System and method for correlating user attention direction and outside view

Abstract

Embodiments are directed towards providing a system that presents customized content to a user of an automobile based on what the user of the automobile is looking at. A first camera captures images of the user of the interior of the automobile, and a second camera captures images of a scene that the user is viewing. The images are analyzed to determine if the user is looking at, or not looking at, an object in the scene. If the user is looking at such an object, content associated with the object is selected and presented to the user.

1 . A system, comprising:
 an output interface that is configured to present content to a person in a vehicle;   a first camera that is configured to capture a plurality of first images of the person;   a second camera that is configured to capture a plurality of second images of a scene that the person is viewing in front of the vehicle;   a memory that is configured to store computer instructions;   a processor that is configured to execute the computer instructions to:
 obtain the first plurality of images captured by the first camera; 
 obtain the second plurality of images captured by the second camera; 
 determine, from a plurality of attention levels, an attention level of the person in an object of interest in the scene in which the person is looking at based on an analysis of an attention position of the person in the plurality of first images and an analysis of a relative position of the object of interest the plurality of second images; 
 determine a type of content from a plurality of types of content based on the determined attention level; 
 select an item of content to present to the person based on the determined type of content and the object of interest that the person is looking at; and 
 present the selected item of content to the person via the output interface. 
   
     
     
         2 . The system of  claim 1 , wherein the processor is configured to determine the attention level by executing further computer instructions to:
 determine an amount of time the person is looking at the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   select the attention level from the plurality of attention levels based on the amount of time the person is looking at the object of interest.   
     
     
         3 . The system of  claim 1 , wherein the processor is configured to determine the attention level by executing further computer instructions to:
 determine a number of times that the person looks at the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   select the attention level from the plurality of attention levels based on the number of times that the person looks at the object of interest.   
     
     
         4 . The system of  claim 1 , wherein the processor is configured to determine the attention level by executing further computer instructions to:
 determine a time sequence of a number of views that the person looks at the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   select the attention level from the plurality of attention levels based on the determined time sequence.   
     
     
         5 . The system of  claim 1 , wherein the processor is configured to determine the attention level by executing further computer instructions to:
 determine a gesture of the person relative to the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   select the attention level from the plurality of attention levels based on the determined gesture.   
     
     
         6 . The system of  claim 1 , wherein the processor is configured to determine the attention level by executing further computer instructions to:
 determine a gesture of the person relative to the object of interest;   determine if another person in the vehicle looks at the object of interest in reaction to the determined gesture of the person; and   select the attention level from the plurality of attention levels based on the other person looking at the object of interest.   
     
     
         7 . The system of  claim 1 , wherein each attention level of the plurality of attention levels corresponds to a different type of content of the plurality of types of content. 
     
     
         8 . The system of  claim 1 , wherein the output interface is a display device configured to display visual content to the person, a speaker configured to output audio content, or a haptic interface configured to provide tactile content to the person. 
     
     
         9 . The system of  claim 1 , wherein the processor is configured to select the item of content by executing further computer instructions to:
 select visual content to be displayed to the person via a display device;   select audio content to be output to the person via a speaker; or   select tactile content to be provided to the person via a haptic interface.   
     
     
         10 . A method, comprising:
 capturing a first plurality of images of a person in a vehicle and a plurality of second images of an area viewable by the person;   determining, from a plurality of attention levels, an attention level of the person in an object of interest in the scene in which the person is looking at based on an analysis of an attention position of the person in the plurality of first images and an analysis of a relative position of the object of interest the plurality of second images;   determining a type of content from a plurality of types of content based on the determined attention level, wherein each attention level of the plurality of attention levels corresponds to a different type of content of the plurality of types of content;   selecting an item of content to present to the person based on the determined type of content and the object of interest that the person is looking at; and   presenting the selected item of content to the person via the output interface.   
     
     
         11 . The method of  claim 10 , wherein determining the attention level comprises:
 determining an amount of time the person is looking at the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   selecting the attention level from the plurality of attention levels based on the amount of time the person is looking at the object of interest.   
     
     
         12 . The method of  claim 10 , wherein determining the attention level comprises:
 determining a number of times that the person looks at the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   selecting the attention level from the plurality of attention levels based on the number of times that the person looks at the object of interest.   
     
     
         13 . The method of  claim 10 , wherein determining the attention level comprises:
 determining a time sequence of a number of views that the person looks at the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   selecting the attention level from the plurality of attention levels based on the determined time sequence.   
     
     
         14 . The method of  claim 10 , wherein determining the attention level comprises:
 determining a gesture of the person relative to the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   selecting the attention level from the plurality of attention levels based on the determined gesture.   
     
     
         15 . The method of  claim 10 , wherein determining the attention level comprises:
 determining a gesture of the person relative to the object of interest;   determining if another person in the vehicle looks at the object of interest in reaction to the determined gesture of the person; and   selecting the attention level from the plurality of attention levels based on the other person looking at the object of interest.   
     
     
         16 . A head unit of an automobile, comprising:
 a display device that is configured to display visual content;   a memory that is configured to store computer instructions; and   a processor that, when executing the computer instructions, causes the head unit to:
 receive a first plurality of images from an interior-facing camera on the automobile; 
 receive a second plurality of images from an exterior-facing camera on the automobile; 
 determine a focus of a user in the automobile on an object of interest based on a direction of the user's attention detected from the plurality of first images and a relative position of the object of interest detected from the plurality of second images; 
 determine, from a plurality of attention levels, an attention level of the user in the object of interest based on the determined focus of the user; 
 determine a type of content from a plurality of types of content based on the determined attention level; 
 select an item of content to present to the user based on the determined type of content and the object of interest that the user is focusing on; and 
 provide the selected item of content to the user via the display device. 
   
     
     
         17 . The head unit of  claim 16 , wherein the processor, when executing the computer instructions to determine the focus of the user, further causes the head unit to:
 determine a focus area of the user based on a position of the user's eyes and head in the first plurality of images; and   identify a location of the object of interest based on a relative position of the object of interest within the second plurality of images.   
     
     
         18 . The head unit of  claim 16 , wherein the processor, when executing the computer instructions to determine the attention level, further causes the head unit to:
 determine an amount of time the person is looking at the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   select the attention level from the plurality of attention levels based on the amount of time the person is looking at the object of interest.   
     
     
         19 . The head unit of  claim 16 , wherein the processor, when executing the computer instructions to determine the attention level, further causes the head unit to:
 determine a number of times that the person looks at the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   select the attention level from the plurality of attention levels based on the number of times that the person looks at the object of interest.   
     
     
         20 . The head unit of  claim 16 , wherein the processor, when executing the computer instructions to determine the attention level, further causes the head unit to:
 determine a time sequence of a number of views that the person looks at the object of interest based on an analysis of the plurality of first images and the plurality of second images; and   select the attention level from the plurality of attention levels based on the determined time sequence.