Eye-tracking enabled wearable devices

Abstract

A system for determining a gaze direction of a user is disclosed. The system may include a first illuminator, a first profile sensor, and at least one processor. The first illuminator may be configured to illuminate an eye of a user. The first profile sensor may be configured to detect light reflected by the eye of the user. The processor(s) may be configured to determine a gaze direction of the user based at least in part on light detected by the first profile sensor.

What is claimed is: 
     
       1. A system for determining a gaze direction of a user, wherein the system comprises:
 a first illuminator configured to illuminate an eye of a user; 
 a first profile sensor configured to detect light reflected by the eye of the user; 
 an area image sensor configured to detect light reflected by the eye of the user, wherein the area image sensor has a sensitive area having a total number of pixels characterized by a total number of rows and a total number of columns; 
 at least one processor configured to determine a gaze direction of the user based at least in part on light detected by the first profile sensor and based at least in part on light detected by the area image sensor; 
 wherein determining the gaze direction of the user based at least in part on light detected by the first profile sensor comprises:
 determining a location of a glint on the eye caused by light reflected by the eye of the user, based at least in part on a higher magnitude light level detected at one or more pixels of the first profile sensor; 
 identifying a subset of a total number of pixels of the first profile sensor which include and surround the one or more pixels of the first profile sensor which detected the higher magnitude light level; and 
 analyzing only the subset of the total number of pixels of the first profile sensor during a subsequent determination of gaze direction; and 
 
 wherein determining the gaze direction of the user based at least in part on light detected by the area image sensor comprises:
 determining magnitudes of light exposure at a subset of the total number of rows of the sensitive area during one or more illumination periods; 
 determining magnitudes of light exposure at a subset of the total number of columns of the sensitive area during one or more illumination periods; 
 identifying a subset of the total number of pixels of the sensitive area based on the magnitudes of light exposure at the subset of the total number of rows and the magnitudes of light exposure at the subset of the total number of columns; and 
 analyzing only the subset of the total number of pixels of the sensitive area during a subsequent illumination period. 
 
 
     
     
       2. The system for determining a gaze direction of a user of  claim 1 , wherein the system further comprises:
 a second profile sensor configured to detect light reflected by the eye of the user, wherein determining the gaze direction of the user is further based at least in part on light detected by the second profile sensor. 
 
     
     
       3. The system for determining a gaze direction of a user of  claim 2 , wherein the at least one processor is further configured to:
 determine a location of a cornea center of the eye based at least in part on light detected by the first profile sensor and light detected by the second profile sensor. 
 
     
     
       4. The system for determining a gaze direction of a user of  claim 1 , wherein the first profile sensor is configured to:
 detect light in two dimensions. 
 
     
     
       5. The system for determining a gaze direction of a user of  claim 2 , wherein:
 the first profile sensor is configured to detect light in one dimension; and 
 the second profile sensor is configured to detect light in one dimension. 
 
     
     
       6. The system for determining a gaze direction of a user of  claim 5 , wherein:
 the first profile sensor is orientated orthogonally to the second profile sensor. 
 
     
     
       7. The system for determining a gaze direction of a user of  claim 5 , wherein the system further comprises:
 a first cylindrical lens between the first profile sensor and the eye; and 
 a second cylindrical lens between the second profile sensor and the eye. 
 
     
     
       8. The system for determining a gaze direction of a user of  claim 1 , wherein the system further comprises:
 a second illuminator configured to illuminate the eye of the user. 
 
     
     
       9. The system for determining a gaze direction of a user of  claim 8 , wherein the at least one processor is further configured to:
 modulate activation of the first illuminator and the second illuminator such that only the first illuminator or the second illuminator is activated at a particular point in time. 
 
     
     
       10. The system for determining a gaze direction of a user of  claim 1 , wherein:
 the area image sensor operates at between about 0.5 and about 10.0 hertz. 
 
     
     
       11. A method for determining a gaze direction of a user, wherein the method comprises:
 illuminating an eye of a user; 
 detecting, with an area image sensor, light reflected by the eye of the user, wherein the area image sensor has a sensitive area having a total number of pixels characterized by a total number of rows and a total number of columns; 
 determining, with at least one processor, a gaze direction of the user based at least in part on light detected by the area image sensor; 
 wherein determining the gaze direction of the user based at least in part on light detected by the area image sensor comprises:
 determining a location of a glint on the eye caused by light reflected by the eye of the user, based at least in part on a higher magnitude light level detected at one or more pixels of the area image sensor; 
 determining magnitudes of light exposure at a subset of the total number of rows during one or more illumination periods; 
 determining magnitudes of light exposure at a subset of the total number of columns during one or more illumination periods; 
 based on the magnitudes of light exposure at the subset of the total number of rows and the magnitudes of light exposure at the subset of the total number of columns, identifying a subset of a total number of pixels of the area image sensor which include and surround the one or more pixels of the area image sensor which detected the higher magnitude light level; and 
 analyzing only the subset of the total number of pixels of the area image sensor during a subsequent determination of gaze direction during a subsequent illumination period. 
 
 
     
     
       12. A non-transitory machine readable medium having instructions stored thereon for determining a gaze direction of a user, wherein the instructions are executable by one or more processors to at least:
 cause an illuminator to illuminate an eye of a user; 
 detect, with an area image sensor, light reflected by the eye of the user, wherein the area image sensor has a sensitive area having a total number of pixels characterized by a total number of rows and a total number of columns; 
 determine a gaze direction of the user based at least in part on light detected by the an area image sensor; 
 wherein determining the gaze direction of the user based at least in part on light detected by the an area image sensor comprises:
 determining a location of a glint on the eye caused by light reflected by the eye of the user, based at least in part on a higher magnitude light level detected at one or more pixels of the an area image sensor; 
 determining magnitudes of light exposure at a subset of the total number of rows during one or more illumination periods; 
 determining magnitudes of light exposure at a subset of the total number of columns during one or more illumination periods; 
 
 based on the magnitudes of light exposure at the subset of the total number of rows and the magnitudes of light exposure at the subset of the total number of columns, identifying a subset of a total number of pixels of the an area image sensor which include and surround the one or more pixels of the profile sensor which detected the higher magnitude light level; and
 analyzing only the subset of the total number of pixels of the an area image sensor during a subsequent determination of gaze direction during a subsequent illumination period. 
 
 
     
     
       13. A system for determining a gaze direction of a user, wherein the system comprises:
 a first illuminator configured to illuminate an eye of a user; 
 an area image sensor configured to detect light reflected by the eye of the user, wherein the area image sensor has a sensitive area having a total number of pixels characterized by a total number of rows and a total number of columns; and 
 at least one processor configured to determine a gaze direction of the user based at least in part on light detected by the area image sensor; 
 wherein determining the gaze direction of the user based at least in part on light detected by the area image sensor comprises:
 determining a location of a glint on the eye caused by light reflected by the eye of the user, based at least in part on a higher magnitude light level detected at one or more pixels of the area image sensor; 
 determining magnitudes of light exposure at a subset of the total number of rows during one or more illumination periods; 
 determining magnitudes of light exposure at a subset of the total number of columns during one or more illumination periods; 
 based on the magnitudes of light exposure at the subset of the total number of rows and the magnitudes of light exposure at the subset of the total number of columns, identifying a subset of a total number of pixels of the area image sensor which include and surround the one or more pixels of the area image sensor which detected the higher magnitude light level; and 
 analyzing only the subset of the total number of pixels of the area image sensor during a subsequent determination of gaze direction during a subsequent illumination period. 
 
 
     
     
       14. The system of  claim 13 , wherein the system further comprises:
 a Fresnel lens in front of a display device; 
 a hot mirror on at least one side of the Fresnel lens or the display device; 
 and wherein:
 the at least one processor is further configured to adjust the trajectory of virtual missiles which are caused to be virtually propelled by the user in virtual space; and 
 
 the first illuminator and image sensor are removable from the system so that they may be used in other devices. 
 
     
     
       15. A system for determining a gaze direction of a user, wherein the system comprises:
 a first illuminator configured to illuminate an eye of a user; 
 a first profile sensor configured to detect light reflected by the eye of the user; 
 an area image sensor configured to detect light reflected by the eye of the user, wherein the area image sensor has a sensitive area having a total number of pixels characterized by a total number of rows and a total number of columns; 
 at least one processor configured to determine a gaze direction of the user based at least in part on light detected by the first profile sensor and based at least in part on light detected by the area image sensor; 
 wherein determining the gaze direction of the user based at least in part on light detected by the first profile sensor comprises:
 determining a location of a glint on the eye caused by light reflected by the eye of the user, based at least in part on a higher magnitude light level detected at one or more pixels of the first profile sensor; 
 identifying a subset of a total number of pixels of the first profile sensor which include and surround the one or more pixels of the first profile sensor which detected the higher magnitude light level; and 
 analyzing only the subset of the total number of pixels of the first profile sensor during a subsequent determination of gaze direction; and 
 
 wherein determining the gaze direction of the user based at least in part on light detected by the area image sensor comprises:
 determining magnitudes of light exposure at a subset of pixels in each of the total number of rows of the sensitive area during one or more illumination periods; 
 determining magnitudes of light exposure at a subset of pixels in each of the total number of columns of the sensitive area during one or more illumination periods; 
 identifying a subset of the total number of pixels of the sensitive area based on the magnitudes of light exposure at the subset of pixels in each of the total number of rows and the magnitudes of light exposure at the subset of pixels in each of the total number of columns; and 
 analyzing only the subset of the total number of pixels of the of the sensitive area during a subsequent illumination period. 
 
 
     
     
       16. A system for determining a gaze direction of a user, wherein the system comprises:
 a first illuminator configured to illuminate an eye of a user, the first illuminator comprising a near-infrared illuminator; 
 a first profile sensor configured to detect light reflected by the eye of the user; 
 at least one processor configured to determine a gaze direction of the user based at least in part on light detected by the first profile sensor; 
 wherein at least the first illuminator and the first profile sensor are disposed within a wearable device comprising a display device and a cold mirror on a back side of the display device, wherein the cold mirror is configured to reflect visible light toward a front side of the display device, and allow near-infrared to pass through the cold mirror, and wherein the first profile sensor is disposed on the opposite side of the cold mirror from the display device; and 
 wherein determining the gaze direction of the user based at least in part on light detected by the first profile sensor comprises:
 determining a location of a glint on the eye caused by light reflected by the eye of the user, based at least in part on a higher magnitude light level detected at one or more pixels of the first profile sensor; 
 identifying a subset of a total number of pixels of the first profile sensor which include and surround the one or more pixels of the first profile sensor which detected the higher magnitude light level; and 
 analyzing only the subset of the total number of pixels of the first profile sensor during a subsequent determination of gaze direction. 
 
 
     
     
       17. The system for determining a gaze direction of a user of  claim 16 , wherein:
 the wearable device further comprises a lens in front of the display device; and 
 the at least one processor is further configured to compensate for the presence of the lens before or during determining the gaze direction of the user. 
 
     
     
       18. A system for determining a gaze direction of a user, wherein the system comprises:
 a first illuminator configured to illuminate an eye of a user; 
 an area image sensor configured to detect light reflected by the eye of the user, wherein the area image sensor has a sensitive area having a total number of pixels characterized by a total number of rows and a total number of columns; and 
 at least one processor configured to determine a gaze direction of the user based at least in part on light detected by the area image sensor; 
 wherein determining the gaze direction of the user based at least in part on light detected by the area image sensor comprises:
 determining a location of a glint on the eye caused by light reflected by the eye of the user, based at least in part on a higher magnitude light level detected at one or more pixels of the area image sensor; 
 determining magnitudes of light exposure at a subset of pixels in each of the total number of rows during one or more illumination periods; 
 determining magnitudes of light exposure at a subset of pixels in each of the total number of columns during one or more illumination periods; 
 based on the magnitudes of light exposure at the subset of pixels in each of the total number of rows and the magnitudes of light exposure at the subset of pixels in each of the total number of columns, identifying a subset of a total number of pixels of the area image sensor which include and surround the one or more pixels of the area image sensor which detected the higher magnitude light level; and 
 
 analyzing only the subset of the total number of pixels of the area image sensor during a subsequent determination of gaze direction during a subsequent illumination period. 
 
     
     
       19. The system of  claim 18 , wherein the system further comprises:
 a Fresnel lens in front of a display device; 
 a hot mirror on at least one side of the Fresnel lens or the display device; 
 and wherein:
 the at least one processor is further configured to adjust the trajectory of virtual missiles which are caused to be virtually propelled by the user in virtual space; and 
 
 the first illuminator and image sensor are removable from the system so that they may be used in other devices.