Method for inferring metal states from eye movements

Abstract

A computer-implemented method infers mental states of a person from eye movements of the person. The method includes identifying elementary features of eye tracker data, such as fixations and saccades, and recognizing from the elementary features a plurality of eye-movement patterns. Each eye-movement pattern is recognized by comparing the elementary features with a predetermined eye-movement pattern template. A given eye-movement pattern is recognized if the elementary features satisfy a set of criteria associated with the template for that eye-movement pattern. The method further includes the step of recognizing from the eye-movement patterns a plurality of eye-behavior patterns corresponding to the mental states of the person. Because high level mental states of the user are determined in real time, the method provides the basis for reliably determining when a user intends to select a target.

What is claimed is:  
     
       1. A computer implemented method for inferring mental states of a person from eye movements of the person in real time, the method comprising: 
       a) identifying a plurality of elementary features of eye tracker data for the person;  
       b) computing from the elementary features of a plurality of eye movement patterns, wherein each pattern satisfies a set of predetermined eye movement pattern template criteria, wherein computing eye movement patterns is performed without requiring any a priori knowledge of contents of the person's visual field; and  
       c) computing from the eye movement patterns a plurality of eye-behavior patterns corresponding to mental states of the person.  
     
     
       2. The method of  claim 1  further comprising classifying the elementary features according to associated eye-behavior patterns. 
     
     
       3. The method of  claim 2  wherein computing the eye movement patterns comprises computing a significant fixation when a current fixation duration is longer than a significant threshold for a current eye-behavior, where the threshold is calculated from recent fixation duration times classified by the current eye-behavior. 
     
     
       4. The method of  claim 1  further comprising computing high level features from the elementary features. 
     
     
       5. The method of  claim 1  wherein the plurality of eye behavior patterns comprises at least three eye behavior patterns. 
     
     
       6. The method of  claim 5  wherein the eye behavior patterns comprise a pattern selected from the group consisting of reading patterns, spacing out patterns, and searching patterns. 
     
     
       7. The method of  claim 1  wherein computing eye movement patterns is performed without requiring knowledge of specific types of objects being displayed in the person's visual field. 
     
     
       8. The method of  claim 1  being documented in a machine-readable code and being stored on a computer storage device. 
     
     
       9. A computer implemented method for inferring mental states of a person from eye movements of the person in real time, the method comprising: 
       a) identifying a plurality of elementary features of eye tracker data for the person;  
       b) computing from the elementary features of a plurality of eye movement patterns, wherein each pattern comprises a temporally ordered sequence of fixations and saccades satisfying a set of predetermined eye movement pattern template criteria; and  
       c) computing from the eye movement patterns a plurality of eye-behavior patterns corresponding to mental states of the person.  
     
     
       10. The method of  claim 9  further comprising classifying the elementary features according to associated eye-behavior patterns. 
     
     
       11. The method of  claim 10  wherein computing the eye movement patterns comprises computing a significant fixation when a current fixation duration is longer than a significant fixation threshold for a current eye-behavior, where the threshold is calculated from recent fixation duration times classified by the current eye-behavior. 
     
     
       12. The method of  claim 9  further comprising computing high level features from the elementary features. 
     
     
       13. The method of  claim 9  wherein the plurality of eye behavior patterns comprises at least three eye behavior patterns. 
     
     
       14. The method of  claim 13  wherein the eye behavior patterns comprise a pattern selected from the group consisting of reading patterns, spacing out patterns, and searching patterns. 
     
     
       15. The method of  claim 9  wherein computing the eye behavior pattern comprises identifying a sequence of short saccades to the right. 
     
     
       16. The method of  claim 9  being documented in a machine-readable code and being stored on a computer storage device. 
     
     
       17. A computer implemented method for inferring from eye movements of a person that the person is reading, the method comprising: 
       a) identifying elementary features of eye tracker data for the person;  
       b) computing from the elementary features a hierarchy of patterns on various interpretive levels, wherein computed patterns on higher levels are derived from computed patterns on lower levels, wherein highest level computed patterns comprise a reading corresponding to a reading state of the person.  
     
     
       18. The method of  claim 17  wherein computing patterns on various interpretive levels comprises identifying a sequence of short saccades to the right. 
     
     
       19. The method of  claim 17  wherein computing patterns on various interpretive levels comprises identifying a plurality of sequences of short saccades to the right, wherein the plurality of sequences are approximately vertically aligned with each other. 
     
     
       20. The method of  claim 17  wherein computing patterns on various interpretive levels and computing highest level patterns is accomplished without requiring any a priori knowledge of the person's visual field. 
     
     
       21. The method of  claim 17  being documented in a machine-readable code and being stored on a computer storage device. 
     
     
       22. An article storing computer- readable instructions that cause one or more hardware devices to:    
         a )  identify a plurality of elementary features of eye tracker data for the person;    
         b )  compute from the elementary features a plurality of eye movement patterns, wherein each pattern satisfies a set of predetermined eye movement pattern template criteria, wherein computing eye movement patterns is performed without requiring any a priori knowledge of contents of the person's visual field; and    
         c )  compute from the eye movement patterns a plurality of eye - behavior patterns corresponding to mental states of the person.    
     
     
       23. The article of  claim 22  further comprising instructions to classify the elementary features according to associated eye- behavior patterns.    
     
     
       24. The method of  claim 23  wherein the instructions to compute the eye movement patterns comprises instructions to compute a significant fixation when a current fixation duration is longer than a significant threshold for a current eye- behavior, where the threshold is calculated from recent fixation duration times classified by the current eye - behavior.    
     
     
       25. The article of  claim 22  further comprising instructions to compute high level features from the elementary features.  
     
     
       26. The article of  claim 22  wherein the plurality of eye behavior patterns comprises at least three eye behavior patterns.  
     
     
       27. The article of  claim 26  wherein the eye behavior patterns comprise a pattern selected from the group consisting of reading patterns, spacing out patterns, and searching patterns.  
     
     
       28. The article of  claim 22  wherein computing eye movement patterns is performed without requiring knowledge of specific types of objects being displayed in the person's visual field.  
     
     
       29. An article storing computer- readable instructions that cause one or more hardware devices to:    
         a )  identify a plurality of elementary features of eye tracker data for the person;    
         b )  compute from the elementary features a plurality of eye movement patterns, wherein each pattern comprises a temporally ordered sequence of fixations and saccades satisfying a set of predetermined eye movement pattern template criteria; and    
         c )  compute from the eye movement patterns a plurality of eye - behavior patterns corresponding to mental states of the person.    
     
     
       30. The article of  claim 29  further comprising instructions to classify the elementary features according to associated eye- behavior patterns.    
     
     
       31. The article of  claim 30  wherein computing the eye movement patterns comprises computing a significant fixation when a current fixation duration is longer than a significant fixation threshold for a current eye- behavior, where the threshold is calculated from recent fixation duration times classified by the current eye - behavior.    
     
     
       32. The article of  claim 29  further comprising computing high level features from the elementary features.  
     
     
       33. The article of  claim 29  wherein the plurality of eye behavior patterns comprises at least three eye behavior patterns.  
     
     
       34. The article of  claim 33  wherein the eye behavior patterns comprise a pattern selected from the group consisting of reading patterns, spacing out patterns, and searching patterns.  
     
     
       35. The article of  claim 29  wherein computing the eye behavior patterns comprises identifying a sequence of short saccades to the right.  
     
     
       36. An article storing computer- readable instructions that cause one or more hardware devices to:    
         a )  identifying elementary features of eye tracker data for the person;    
         b )  compute from the elementary features a hierarchy of patterns on various interpretive levels, wherein computed patterns on higher levels are derived from computed patterns on lower levels, wherein highest level computed patterns comprise a reading pattern corresponding to a reading state of the person.    
     
     
       37. The article of  claim 36  wherein computing patterns on various interpretive levels comprises identifying a sequence of short saccades to the right.  
     
     
       38. The article of  claim 36  wherein computing patterns on various interpretive levels comprises identifying a plurality of sequences of short saccades to the right, wherein the plurality of sequences are approximately vertically aligned with each other.  
     
     
       39. The article of  claim 36  wherein computing patterns on various interpretive levels and computing highest level patterns is accomplished without requiring any a priori knowledge of the person's visual field.