Estimation of absolute depth from polarization measurements

Abstract

A head mounted display comprises an eye tracking system configured to enable eye tracking using polarization. The eye tracking system includes one or more illumination sources and an optical detector comprising polarization sensitive pixels. The one or more illumination sources are configured to illuminate a user's eye and generate reflections directed towards the optical detector. The eye tracking system determines, for each polarization sensitive pixel in a subset of the polarization sensitive pixels, one or more estimation parameters. The eye tracking system determines, for the subset of the polarization sensitive pixels, depth information for one or more glints associated with one or more surfaces of the eye, based in part on the polarization of the reflections and the one or more estimation parameters. The determined depth information is used to update a model of the eye. The eye tracking system determines eye tracking information based on the updated model.

What is claimed is: 
     
       1. A head-mounted display (HMD) comprising:
 an eye tracking system that includes:
 one or more illumination sources configured to illuminate an eye of a user with light, and 
 an optical detector comprising an array of a plurality of polarization sensitive pixels configured to:
 capture one or more images of the eye, and 
 determine intensity and polarization information associated with the one or more images; and 
 
 a controller configured to:
 determine, for each polarization sensitive pixel in a subset of the plurality of polarization sensitive pixels, one or more estimation parameters associated with the eye tracking system, 
 determine, for the subset of the plurality of polarization sensitive pixels, depth information for one or more glints associated with one or more surfaces of a plurality of surfaces of the eye, based in part on the determined polarization information and the one or more estimation parameters, 
 update a model of the eye based in part on the determined depth information, and 
 determine eye tracking information associated with the updated model of the eye, the eye tracking information including an eye orientation of the eye. 
 
 
 
     
     
       2. The HMD of  claim 1 , wherein the controller is further configured to:
 update the model of the eye by fitting one or more model components of a plurality of model components in the model to the one or more surfaces of the eye, using the depth information for the one or more glints. 
 
     
     
       3. The HMD of  claim 2 , wherein the controller is further configured to:
 determine the eye tracking information based in part on the fitted one or more model components. 
 
     
     
       4. The HMD of  claim 1 , wherein:
 each polarization sensitive pixel in the plurality of polarization sensitive pixels includes a three or more native pixels; and 
 each native pixel of the three or more native pixels comprises a detector coupled to a polarization filter. 
 
     
     
       5. The HMD of  claim 1 , wherein the controller is further configured to:
 determine a plurality of surface normals associated with the subset of the plurality of polarization sensitive pixels based on the determined intensity and polarization information; and determine a three-dimensional (3D) shape of the eye based in part on the determined surface normals. 
 
     
     
       6. The HMD of  claim 1 , wherein:
 the optical detector is configured to capture, for at least three polarization angles, an intensity of light reflected from at least one surface of the eye; and 
 the controller is further configured to determine a degree of polarization of the reflected light based in part on the captured intensity. 
 
     
     
       7. The HMD of  claim 6 , wherein the controller is further configured to:
 determine a zenith angle associated with the reflected light based in part on the determined degree of polarization; and 
 determine depth information for a glint of the one or more glints, based on the one or more estimation parameters and the zenith angle. 
 
     
     
       8. The HMD of  claim 6 , wherein the one or more estimation parameters determined for each polarization sensitive pixel in the subset include: a displacement between one of the one or more illumination sources and that polarization sensitive pixel, and an angle between a ray of light reflected from a surface of the eye and an axis of that polarization sensitive pixel capturing the ray of reflected light. 
     
     
       9. The HMD of  claim 8 , wherein the controller is further configured to:
 determine a zenith angle associated with the reflected light based in part on the determined degree of polarization; and 
 determine depth information for a glint of the one or more glints based on the displacement, the angle and the zenith angle. 
 
     
     
       10. The HMD of  claim 1 , wherein the controller is further configured to:
 retrieve one or more model parameters associated with a stored model of the eye; 
 extract one or more model parameter values from the determined depth information; 
 update at least one of the one or more model parameters by replacing one or more stored model parameter values with the one or more extracted model parameter values, based in part on a determination of a difference between at least one of the one or more stored model parameter values and the one or more extracted model parameter values; and 
 store the updated model in a data store associated with the eye tracking module. 
 
     
     
       11. A head-mounted display (HMD) comprising:
 a display unit including one or more illumination sources, the display unit configured to emit content and illuminate an eye of a user, 
 an eye tracking system that includes:
 an optical detector comprising an array of a plurality of polarization sensitive pixels configured to:
 capture one or more images of the eye, and 
 determine intensity and polarization information associated with the one or more images; and 
 
 a controller configured to:
 determine, for each polarization sensitive pixel in a subset of the plurality of polarization sensitive pixels, one or more estimation parameters associated with the eye tracking system, 
 determine, for the subset of the plurality of polarization sensitive pixels, depth information for one or more glints associated with one or more surfaces of a plurality of surfaces of the eye, based in part on the determined polarization information, one or more distances between the optical detector and the one or more illumination sources, and the one or more estimation parameters, 
 update a model of the eye based in part on the determined depth information, and 
 determine eye tracking information associated with the updated model of the eye, the eye tracking information including an eye orientation of the eye. 
 
 
 
     
     
       12. The HMD of  claim 11 , wherein the controller is further configured to:
 update the model of the eye by fitting one or more model components of a plurality of model components in the model to the one or more surfaces of the eye, using the depth information for the one or more glints. 
 
     
     
       13. The HMD of  claim 12 , wherein the controller is further configured to:
 determine the eye tracking information based in part on the fitted one or more model components. 
 
     
     
       14. The HMD of  claim 11 , wherein the controller is further configured to:
 determine a plurality of surface normals associated the subset of the plurality of polarization sensitive pixels based on the determined intensity and polarization information; and 
 determine a three-dimensional (3D) shape of the eye based in part on the determined surface normals. 
 
     
     
       15. The HMD of  claim 11 , wherein:
 the optical detector is configured to capture, for at least three polarization angles, an intensity of light reflected from at least one surface of the eye; and 
 the controller is further configured to determine a degree of polarization of the reflected light based in part on the captured intensity. 
 
     
     
       16. The HMD of  claim 15 , wherein the controller is further configured to:
 determine a zenith angle associated with the reflected light based in part on the determined degree of polarization; and 
 determine depth information for a glint of the one or more glints based on the one or more estimation parameters and the zenith angle. 
 
     
     
       17. The HMD of  claim 15 , wherein the one or more estimation parameters determined for each polarization sensitive pixel in the subset include: a displacement between one of the one or more illumination sources and that polarization sensitive pixel, and an angle between a ray of light reflected from a surface of the eye and an axis of that polarization sensitive pixel capturing the ray of reflected light. 
     
     
       18. The HMD of  claim 17 , wherein the controller is further configured to:
 determine a zenith angle associated with the reflected light based in part on the determined degree of polarization; and 
 determine depth information for a glint of the one or more glints based on the displacement, the angle and the zenith angle. 
 
     
     
       19. A head-mounted display (HMD) comprising:
 an eye tracking system that includes:
 one or more illumination sources configured to illuminate an eye of a user with unpolarized light, and 
 an optical detector comprising an array of a plurality of polarization sensitive pixels coupled to a dynamic polarizer, each of the polarization sensitive pixels configured to:
 capture one or more images of the eye, and 
 determine intensity and polarization information associated with the one or more images; and 
 
 a controller configured to:
 determine, for each polarization sensitive pixel in a subset of the plurality of polarization sensitive pixels, one or more estimation parameters associated with the eye tracking system, 
 determine, for the subset of the plurality of polarization sensitive pixels, depth information for one or more glints associated with one or more surfaces of a plurality of surfaces of the eye, based in part on the determined polarization information, one or more distances between the optical detector and the one or more illumination sources, and the one or more estimation parameters, 
 update a model of the eye based in part on the determined depth information, and 
 determine eye tracking information associated with the updated model of the eye, the eye tracking information including an eye orientation of the eye. 
 
 
 
     
     
       20. The HMD of  claim 19 , wherein the dynamic polarizer is an optical filter that is permeable to light waves of three or more polarization states.