Eye tracking system and methods of using an eye tracking system

Abstract

A method for determining a gaze convergence function for a user of an eye tracking system. The method comprising presenting a first calibration object to the user on one or more display screens, wherein the first calibration object appears to be at a known first object distance from the user. While the first calibration object is being presented, the method: captures a calibration images of the user's eyes and calculates a first interpupillary distance, IPD, value. The method performs similar processing steps for a second calibration object to calculate a second IPD value. The method then determines a gaze convergence function which defines a relationship between IPD values and gaze convergence distances based on: i) the first IPD value and the associated first object distance; and ii) the second IPD value and the associated second object distance.

1 . A method for determining a gaze convergence function for a user of an eye tracking system, the method comprising:
 presenting a first calibration object to the user on one or more display screens, wherein the first calibration object appears to be at a known first object distance from the user;   while the first calibration object is being presented:
 capturing a calibration image of the user's left eye and determining a first left pupil centre value, which represents the centre of the pupil of the user's left eye as a 3-dimensional coordinate, 3D, based on the captured calibration image of the user's left eye; 
 capturing a calibration image of the user's right eye and determining a first right pupil centre value, which represents the centre of the pupil of the user's right eye as a 3-dimensional coordinate, based on the captured calibration image of the user's right eye; 
   calculating a first interpupillary distance, IPD, value as the distance between the first left pupil centre value and the first right pupil centre value;   presenting a second calibration object to the user on one or more display screens, wherein the second calibration object appears to be at a known second object distance from the user;   while the second calibration object is being presented:
 capturing a calibration image of the user's left eye and determining a second left pupil centre value, which represents the centre of the pupil of the user's left eye as a 3-dimensional coordinate, based on the captured calibration image of the user's left eye; 
 capturing a calibration image of the user's right eye and determining a second right pupil centre value, which represents the centre of the pupil of the user's right eye as a 3-dimensional coordinate, based on the captured calibration image of the user's right eye; 
   calculating a second IPD value as the distance between the second left pupil centre value and the second right pupil centre value;   determining a gaze convergence function which defines a relationship between IPD values and gaze convergence distances based on: i) the first IPD value and the associated first object distance; and ii) the second IPD value and the associated second object distance.   
     
     
         2 . The method of  claim 1 , further comprising:
 presenting one or more further calibration objects to the user on one or more display screens, wherein each of the one or more further calibration objects appears to be at a known further object distance from the user;   while each of the further calibration objects is being presented:
 capturing a calibration image of the user's left eye and determining a further left pupil centre value, which represents the centre of the pupil of the user's left eye as a 3-dimensional coordinate, based on the captured calibration image of the user's left eye; 
 capturing a calibration image of the user's right eye and determining a further right pupil centre value, which represents the centre of the pupil of the user's right eye as a 3-dimensional coordinate, based on the captured calibration image of the user's right eye; 
   calculating a further IPD value for each of the further calibration objects as the distance between the further first left pupil centre value and the further first right pupil centre value for the calibration images that were captured with the respective one or more further calibration objects being presented; and   determining the gaze convergence function also based on, for each of the one or more further calibration objects: i) the further IPD value and the associated further object distance.   
     
     
         3 . The method of  claim 1 , wherein the gaze convergence function defines a linear relationship between: i) the inverse of IPD values; and ii) the inverse of gaze convergence distances. 
     
     
         4 . The method of  claim 1 , wherein capturing a calibration image of the user's eye and determining a pupil centre value comprises:
 simultaneously capturing a first calibration image and a second calibration image of the user's eye from different cameras and determining the pupil centre value, which represents the centre of the pupil of the user's eye as a 3-dimensional coordinate, based on the first and second calibration images.   
     
     
         5 . The method of any one of  claim 1 , further comprising:
 while the calibration objects are being presented:
 illuminating the user's left eye with a plurality of illuminators and capturing a calibration image of the user's left eye that includes reflections of light provided by the plurality of illuminators as a plurality of glints, determining a 3-dimensional position of the cornea of the user's left eye based on the plurality of glints, and determining the left pupil centre value based on: i) the captured calibration image of the user's left eye; and ii) the determined 3-dimensional position of the cornea of the user's left eye; 
 illuminating the user's right eye with a plurality of illuminators and capturing a calibration image of the user's right eye that includes reflections of light provided by the plurality of illuminators as a plurality of glints, determining a 3-dimensional position of the cornea of the user's right eye based on the plurality of glints, and determining the right pupil centre value based on: i) the captured calibration image of the user's right eye; and ii) the determined 3-dimensional position of the cornea of the user's right eye. 
   
     
     
         6 . The method of any  claim 1 , further comprising:
 while the first calibration object is being presented, determining a first gaze angle for the user;   while the second calibration object is being presented, determining a second gaze angle for the user; and   wherein determining the gaze convergence function is based on: i) the first IPD value, the associated first object distance, and the determined first gaze angle; and ii) the second IPD value, the associated second object distance, and the determined second gaze angle.   
     
     
         7 . The method of  claim 6 , wherein the gaze convergence function defines the gaze convergence distance as a function of IPD value and gaze angle. 
     
     
         8 . The method of  claim 6 , further comprising:
 determining a plurality of gaze convergence functions, each gaze convergence function being applicable for different gaze angles, wherein each of the gaze convergence functions defines the gaze convergence distance as a function of IPD value.   
     
     
         9 . The method of  claim 8 , wherein each of the gaze convergence functions is applicable for a different range of gaze angles, and optionally wherein the range of gaze angles are offset from each other in a horizontal plane and/or a vertical plane. 
     
     
         10 . The method of  claim 6 , further comprising:
 determining a plurality of gaze convergence functions, each gaze convergence function being applicable for a different gaze zone, wherein each of the gaze convergence functions defines the gaze convergence distance as a function of IPD value.   
     
     
         11 . The method of  claim 1 , wherein the eye tracking system is a head-mounted eye tracking system. 
     
     
         12 . A method of determining a gaze convergence distance for a user of an eye tracking system, the method comprising:
 capturing an image of the user's left eye and determining a left pupil centre value, which represents the centre of the pupil of the user's left eye as a 3-dimensional coordinate;   capturing an image of the user's right eye and determining a right pupil centre value, which represents the centre of the pupil of the user's right eye as a 3-dimensional coordinate;   calculating an interpupillary distance, IPD, value as the distance between the left pupil centre value and the right pupil centre value;   applying a gaze convergence function to the calculated IPD value to determine a gaze convergence distance, wherein the gaze convergence function defines a relationship between IPD values and gaze convergence distances.   
     
     
         13 . The method of  claim 12 , further comprising:
 using the determined gaze convergence distance to control the focal length of a varifocal lens of the eye tracking system.   
     
     
         14 . The method of  claim 12 , wherein the eye tracking system is an extended reality system. 
     
     
         15 . The method of any one of  claim 12 , further comprising:
 presenting a plurality of selectable objects to the user on one or more display screens, wherein the first calibration object, wherein each of the plurality of selectable objects is at a known object distance from the user and are at a known location on the on one or more display screens;   determining a gaze angle for the user at the same time as the images of the user's left and right eyes are captured; and   using the determined gaze convergence distance in combination with the determined gaze angle to identify one of the selectable objects as a selected object.   
     
     
         16 . The method of any one of  claim 12 , further comprising determining the gaze convergence function for the user of an eye tracking system, wherein determining the gaze convergence function for the user of an eye tracking system comprises:
 presenting a first calibration object to the user on one or more display screens, wherein the first calibration object appears to be at a known first object distance from the user;   while the first calibration object is being presented:
 capturing a calibration image of the user's left eye and determining a first left pupil centre value, which represents the centre of the pupil of the user's left eye as a 3-dimensional coordinate, based on the captured calibration image of the user's left eye; 
 capturing a calibration image of the user's right eye and determining a first right pupil centre value, which represents the centre of the pupil of the user's right eye as a 3-dimensional coordinate, based on the captured calibration image of the user's right eye; 
   calculating a first interpupillary distance, IPD, value as the distance between the first left pupil centre value and the first right pupil centre value;   presenting a second calibration object to the user on one or more display screens, wherein the second calibration object that appears to be at a known second object distance from the user;   while the second calibration object is being presented:
 capturing a calibration image of the user's left eye and determining a second left pupil centre value, which represents the centre of the pupil of the user's left eye as a 3-dimensional coordinate, based on the captured calibration image of the user's left eye; 
 capturing a calibration image of the user's right eye and determining a second right pupil centre value, which represents the centre of the pupil of the user's right eye as a 3-dimensional coordinate, based on the captured calibration image of the user's right eye; 
   calculating a second IPD value as the distance between the second left pupil centre value and the second right pupil centre value;   determining a gaze convergence function based on: i) the first IPD value and the associated first object distance; and ii) the second IPD value and the associated second object distance.