Artificial intelligence device capable of being controlled according to user's gaze and method of operating the same

Abstract

An artificial intelligence (AI) device capable of being controlled according to a user's gaze includes a communication unit, a camera configured to capture an image of a user, and a processor configured to acquire user state information from the image of the user, acquire a gaze position of the user based on the acquired user state information, calculate a distance between the acquired gaze position and the camera, receive, from one or more external AI devices, one or more distances between gaze positions of the user respectively acquired by the external AI devices and cameras respectively provided in the external AI devices through the communication unit, and compare the calculated distance with the received one or more distances to select a controlled device.

What is claimed is: 
     
       1. An artificial intelligence (AI) device comprising:
 a communication circuit; 
 a camera; and 
 one or more processors configured to: 
 acquire user state information from one or more images of a user captured via the camera; 
 acquire a gaze position of the user based on the acquired user state information; 
 calculate a first distance between the acquired gaze position and the camera; 
 receive, from at least one external AI device via the communication circuit, a respective calculated distance between the gaze position of the user and a corresponding camera of each of the at least one external AI device; and 
 select a device as a controlled device based on a comparison of the first calculated distance and the respective calculated distances of each of the at least one external AI device. 
 
     
     
       2. The AI device of  claim 1 , wherein a device having a minimum distance among the first calculated distance and the respective calculated distances is selected as the controlled device. 
     
     
       3. The AI device of  claim 2 , wherein the one or more processors are further configured to transmit to the at least one external AI device a signal indicating that the AI device is selected as the controlled device and that an operation state of the at least one external AI device is to be changed to an inactive state based on the AI device having the minimum distance. 
     
     
       4. The AI device of  claim 2 , wherein the one or more processors are further configured to transmit to one the at least one external AI device a signal indicating that the one external AI device is selected as the controlled device based on the one external AI device having the minimum distance. 
     
     
       5. The AI device of  claim 1 , wherein the user state information includes a face direction, a face angle, and a gaze direction of the user. 
     
     
       6. The AI device of  claim 5 , further comprising a memory, wherein:
 the one or more processors are further configured to acquire the gaze position using the user state information as an input to a gaze position recognition model stored in the memory; and 
 the gaze position recognition model is an artificial neural network. 
 
     
     
       7. The AI device of  claim 6 , wherein:
 training data of the gaze position recognition model includes the user state information and gaze coordinates as labeling data; and 
 the gaze position recognition model is trained such that a difference between an actual position of the gaze and gaze coordinates determined by the gaze position recognition model is minimized. 
 
     
     
       8. The AI device of  claim 6 , wherein the gaze position is indicated by three-dimensional gaze coordinates, and the one or more processors are further configured to calculate a distance between coordinates corresponding to the camera and the gaze coordinates, wherein the coordinates corresponding to the camera are represented as an origin. 
     
     
       9. The AI device of  claim 3 , further comprising a microphone, wherein the one or more processors are further configured to acquire an intention of a voice command of a user received via the microphone and cause an operation to be performed based on the acquired intention. 
     
     
       10. The AI device of  claim 1 , wherein the one or more processors are further configured to calculate the first distance between the acquired gaze position and the camera after recognizing an eyeblink of the user based on the one or more images of the user captured via the camera. 
     
     
       11. A method of operating an artificial intelligence (AI) device, the method comprising:
 capturing one or more images of a user; 
 acquiring user state information from the one or more images of the user; 
 acquiring a gaze position of the user based on the acquired user state information; 
 calculating a first distance between the acquired gaze position and the camera; 
 receiving, from at least one external AI device, a respective calculated distance between the gaze position of the user and a corresponding camera of each of the at least one external AI device; and 
 selecting a device as a controlled device based on a comparison of the first calculated distance and the respective calculated distances of each of the at least one external AI device. 
 
     
     
       12. The method of  claim 11 , wherein a device having a minimum distance among the first calculated distance and the respective calculated distances is selected as the controlled device. 
     
     
       13. The method of  claim 12 , further comprising transmitting to the at least one external AI device a signal indicating that the AI device is selected as the controlled device and that an operation state of the at least one external AI device is to be changed to an inactive state based on the AI device having the minimum distance. 
     
     
       14. The method of  claim 12 , further comprising transmitting to one of the at least one external AI device a signal indicating that the one external AI device is selected as the controlled device based on the one external AI device having the minimum distance. 
     
     
       15. The method of  claim 11 , wherein:
 the user state information includes a face direction, a face angle, and a gaze direction of the user; 
 the gaze position is acquired using the user state information and a gaze position recognition model; and 
 the gaze position recognition model is an artificial neural network.