Systems and methods for biomechanically-based eye signals for interacting with real and virtual objects

Abstract

Systems and methods are provided for discerning the intent of a device wearer primarily based on movements of the eyes. The system can be included within unobtrusive headwear that performs eye tracking and controls screen display. The system can also utilize remote eye tracking camera(s), remote displays and/or other ancillary inputs. Screen layout is optimized to facilitate the formation and reliable detection of rapid eye signals. The detection of eye signals is based on tracking physiological movements of the eye that are under voluntary control by the device wearer. The detection of eye signals results in actions that are compatible with wearable computing and a wide range of display devices.

1 - 30 . (canceled) 
     
     
         31 . A method for providing a graphical user interface to determine intent of a user based at least in part on movement of the user's one or both eyes using a detector, comprising:
 identifying, with the detector, when the user's one or both eyes are directed at an object at an object location;   identifying, with the detector, a first saccade of the user's one or both eyes from the object location towards a target at a target location;   identifying, with the detector, one or more corrective saccades of the user's one or both eyes moving closer towards the target location;   confirming, with the detector, that at least one of the corrective saccades is completed within a pre-determined distance from the target location; and   performing an action related to one or more of the object, the object location, the target, and the target location.   
     
     
         32 . The method of  claim 31 , wherein identifying, with the detector, one or more corrective saccades of the user's one or both eyes moving closer towards the target location comprises one or more of:
 identifying a first length of the first saccade and determining that a second saccade has a smaller length than the first saccade;   determining when the user's one or both eyes moving closer towards the target location has an angular velocity greater than a pre-determined lower velocity threshold and less than a predetermined upper velocity threshold; and   identifying a second length of a second saccade and determining when the second length is greater than a pre-determined lower length threshold and less than a predetermined upper length threshold.   
     
     
         33 . The method of  claim 31 , wherein the first saccade is a memory-guided saccade. 
     
     
         34 . A method for providing a graphical user interface to determine intent of a user based at least in part on movement of the user's one or both eyes using a detector, comprising:
 identifying, with the detector, when the user's one or both eyes are directed at a first object at a first object location;   identifying, with the detector, a saccade of the user's one or both eyes from the first object location towards a target object at a target location on a display;   confirming, with the detector, that one of a predicted landing location and an actual landing location of the saccade is within a pre-determined distance from the target location; and   performing an action that includes replacing the target object with an icon at the target location related to one of the first object, and the first object location.   
     
     
         35 . The method of  claim 34 , wherein one or more additional actions are performed upon confirming that one of a predicted landing location and a measured landing location of the saccade is within a pre-determined distance from the target location. 
     
     
         36 . The method of  claim 34 , wherein the saccade is a memory-guided saccade. 
     
     
         37 . The method of  claim 34 , wherein the icon comprises one of the first object, a thumbnail of the first object, a likeness of the first object, a portion of text in a region of the first object location, an image of the first object location, and a portion of an image of the first object location. 
     
     
         38 . The method of  claim 34 , wherein the icon comprises one of the first object, and the first object location familiar to the user, thereby reducing a cognitive load of the user. 
     
     
         39 . The method of  claim 38 , whereby reducing the user's cognitive load reduces the time required to produce an additional intent by the user. 
     
     
         40 . The method of  claim 34 , wherein replacing the target object with an icon is performed prior to completion of the saccade. 
     
     
         41 . The method of  claim 34 , wherein replacing the target object with an icon is performed prior to completion of the saccade to produce a gap effect, thereby reducing time required to produce an additional intent by the user. 
     
     
         42 . The method of  claim 34 , wherein the action is performed without waiting for perception of the target object. 
     
     
         43 . A method for providing a graphical user interface to determine intent of a user based at least in part on movement of the user's head and one or both of the user's eyes using a head movement detector and an eye movement detector, comprising:
 identifying, with the head movement detector, when the user's head moves at a head velocity;   identifying at substantially the same time, with the eye movement detector, when the one or both of the user's eyes move at an eye velocity;   identifying, based at least in part on the head velocity and the eye velocity, a vestibulo-ocular movement of the one or both of the user's eyes;   confirming that the one or both of the user's eyes are directed at a viewed object at a viewed object location based at least in part on the determined vestibulo-ocular movement of the one or both of the user's eyes; and   performing an action related to one or more of the viewed object, and the viewed object location.   
     
     
         44 . The method of  claim 43 , wherein identifying the vestibulo-ocular eye movement comprises:
 calculating, based on the head velocity, a head direction and a head magnitude;   calculating, based on the eye velocity, an eye direction and an eye magnitude; and   confirming that the head direction is opposite the eye direction within a predetermined direction threshold, and the head magnitude and the eye magnitude are similar within a predetermined magnitude threshold.   
     
     
         45 . The method of  claim 44 , wherein the action is further related to one or more of the eye direction, the head direction, the eye magnitude, and the head magnitude. 
     
     
         46 . The method of  claim 43 , wherein the head movement detector is one of a scene camera affixed to a headwear worn by the user and directed at the environment of the user, and an accelerometer affixed to the headwear. 
     
     
         47 . The method of  claim 43 , wherein the eye movement detector comprises one or more cameras affixed to a headwear worn by the user and directed toward the one or both of the user's eyes. 
     
     
         48 . The method of  claim 43 , wherein the viewed object is one of a virtual object shown on a display, and a real object. 
     
     
         49 . The method of  claim 48 , wherein the display is one of an augmented reality display, a virtual reality display, a mobile display device, a sign, and a display monitor. 
     
     
         50 . The method of  claim 49 , wherein the action includes showing an indicator on the display within a predetermined viewing distance from the viewed object location. 
     
     
         51 . The method of  claim 50 , wherein the indicator is one of one or more alphanumeric characters, halos, cursors, arrows, and symbols. 
     
     
         52 . A method for providing a graphical user interface to determine intent of a user based at least in part on movements of the user's head and one or both of the user's eyes using a head movement detector and an eye movement detector, comprising:
 identifying, with the eye movement detector, an initial viewed object at an initial viewed object location;   identifying, with the head movement detector, when the user's head moves at a head velocity;   identifying at substantially the same time, with the eye movement detector, when the one or both of the user's eyes move at an eye velocity;   identifying, based on the eye velocity, a first saccadic movement of the eye;   confirming, either upon completion of the head movement and the first saccadic eye movement or before completion of the head movement and the first saccadic eye movement, that the one or both of the user's eyes are directed at a target object at a target object location; and   performing an action related to one or more of the initial viewed object, the initial viewed object location, the target object, and the target object location.   
     
     
         53 . The method of  claim 52 , wherein the action is performed without waiting for perception of the target object by the user. 
     
     
         54 . The method of  claim 55 , wherein the action is performed within 250 milliseconds after completion of the first saccadic movement. 
     
     
         55 . The method of  claim 52 , wherein identifying the first saccade comprises determining when the eye velocity is greater than a predetermined velocity threshold. 
     
     
         56 . The method of  claim 52 , wherein the initial viewed object is one of a real object, and a virtual object shown on a display. 
     
     
         57 . The method of  claim 52 , wherein the target object is one of a real object, and a virtual object shown on a display. 
     
     
         58 . The method of  claim 52 , wherein the target object is virtual object shown on a display and wherein the action includes removing the target object from the display. 
     
     
         59 . The method of  claim 52 , wherein the action includes showing one or more additional target virtual objects on a display. 
     
     
         60 . The method of  claim 52 , further comprising:
 identifying, with the eye movement detector, one or more corrective saccadic eye movements following the first saccadic eye movement;   confirming, either upon completion of the one or more corrective saccadic eye movements or before completion of the one or more corrective saccadic eye movements, that the at least one eye of the user is directed at a corrected target object at a corrected target object location; and   performing an action related to one of the corrected target object, and the corrected target object location.