Efficient gesture processing

Abstract

Embodiments of the invention describe a system to efficiently execute gesture recognition algorithms. Embodiments of the invention describe a power efficient staged gesture recognition pipeline including multimodal interaction detection, context based optimized recognition, and context based optimized training and continuous learning. Embodiments of the invention further describe a system to accommodate many types of algorithms depending on the type of gesture that is needed in any particular situation. Examples of recognition algorithms include but are not limited to, HMM for complex dynamic gestures (e.g. write a number in the air), Decision Trees (DT) for static poses, peak detection for coarse shake/whack gestures or inertial methods (INS) for pitch/roll detection.

1 . An article of manufacture comprising a machine-readable storage medium that provides instructions that, if executed by a machine, will cause the machine to perform operations comprising:
 receiving data from a sensor indicating a motion, the sensor having an accelerometer;   determining, via a first set of one or more algorithms, whether the motion is a gestural motion based on at least one of time duration of the data and an energy level of the data; and   determining, via a second set of one or more algorithms, a candidate gesture based on the data in response to determining the motion is a gestural motion, the second set of algorithm(s) to include a gesture recognition algorithm.   
     
     
         2 . The article of manufacture of  claim 1 , the operations further comprising:
 discarding the data in response to determining the motion is not a gestural motion.   
     
     
         3 . The article of manufacture of  claim 1 , wherein the first set of algorithm(s) includes one or more low-complexity algorithms and the machine includes a low-power processor and a main processing unit, the first set of algorithm(s) to be executed via the low-power processor and the second set of algorithm(s) to be executed via the main processing unit. 
     
     
         4 . The article of manufacture of  claim 1 , wherein the gesture recognition algorithm is based on a Hidden Markov Model (HMM). 
     
     
         5 . The article of manufacture of  claim 4 , wherein determining a candidate gesture comprises:
 using context of a system user to select a subset of one or more allowed gestures; and   restricting gesture models loaded by the HMM algorithm to the subset of allowed gesture(s).   
     
     
         6 . The article of manufacture of  claim 4 , wherein determining a candidate gesture comprises:
 using context of a system user to reject a subset of one or more disallowed gestures; and   selecting an HMM Filler model that discards the subset of disallowed gesture(s).   
     
     
         7 . The article of manufacture of  claim 4 , wherein an HMM training set and one or more gesture models is based on physical activity of a user of the machine. 
     
     
         8 . The article of manufacture of  claim 4 , the gesture rejection algorithm to validate a gesture recognized by the HMM algorithm by comparing duration and energy of the gestural motion with one or more of a minimum and a maximum value of duration and energy obtained from a database of training data. 
     
     
         9 . The article of manufacture of  claim 1 , wherein the sensor is included in the machine, the machine comprises a mobile device, and determining, via the first algorithm(s), whether the motion is a gestural motion is further based on at least one of a user context of the mobile device, and an explicit action from the user to indicate a period of gesture commands. 
     
     
         10 . The article of manufacture of  claim 1 , wherein determining a candidate gesture based on the data comprises
 accessing a database of one or more example gesture inputs, the example gesture input(s) to include a minimum and a maximum time duration; and   verifying the time duration of the gestural motion is within the minimum and maximum time durations of an example gesture input.   
     
     
         11 . The article of manufacture of  claim 1 , wherein the data from the sensor is included in a series of data segments, one or more segments to indicate a motion defined by an energy threshold, and receiving the data from the sensor is in response to the data exceeding the energy threshold. 
     
     
         12 . An article of manufacture comprising a machine-readable storage medium that provides instructions that, if executed by a machine, will cause the machine to perform operations comprising:
 receiving data from a sensor indicating a motion, the sensor having an accelerometer;   determining a subset of one or more gesture recognition algorithms from a plurality of gesture recognition algorithms based, at least in part, on one or more signal characteristics of the data; and   determining a gesture from the data from the sensor based, at least in part, on applying the subset of gesture recognition algorithm(s) to the data.   
     
     
         13 . The article of manufacture of  claim 12 , wherein the signal characteristic(s) of the data comprise an energy magnitude of the data. 
     
     
         14 . The article of manufacture of  claim 13 , wherein determining the subset of gesture recognition algorithms is based, at least in part, on comparing the energy magnitude of the data with one or more magnitude values associated with one of the plurality of gesture algorithms. 
     
     
         15 . The article of manufacture of  claim 12 , wherein the signal characteristic(s) of the data comprise a time duration of the data. 
     
     
         16 . The article of manufacture of  claim 15 , wherein determining the subset of gesture recognition algorithms is based, at least in part, on comparing the time duration of the data with one or more time values associated with one of the plurality of gesture algorithms. 
     
     
         17 . The article of manufacture of  claim 12 , wherein the signal characteristic(s) of the data comprise a frequency spectrum of the data. 
     
     
         18 . The article of manufacture of  claim 17 , wherein determining the subset of gesture recognition algorithms is based, at least in part, on comparing the frequency spectrum of the data with one or more spectrum patterns stored associated with one of the plurality of gesture algorithms. 
     
     
         19 . A method comprising:
 receiving data from a sensor indicating a motion, the sensor having an accelerometer;   determining, via a first set of one or more algorithms, whether the motion is a gestural motion based on at least one of time duration of the data and an energy level of the data; and   determining, via a second set of one or more algorithms, a candidate gesture based on the data in response to determining the motion is a gestural motion, the second set of algorithm(s) to include a gesture recognition algorithm.   
     
     
         20 . The method of  claim 19 , the first set of algorithm(s) to comprise one or more low-complexity algorithms to be executed via a low-power processor and the second set of algorithm(s) to be executed via a main processing unit. 
     
     
         21 . The method of  claim 19 , wherein the second set of algorithm(s) includes a Hidden Markov Model (HMM) gesture rejection algorithm and determining a candidate gesture comprises:
 using context of a system user to select a subset of one or more allowed gestures; and   restricting gestures used by the HMM algorithm to the subset of allowed gesture(s).   
     
     
         22 . A method comprising:
 receiving data from a sensor indicating a motion, the sensor having an accelerometer;   determining a subset of one or more gesture recognition algorithms from a plurality of gesture recognition algorithms based, at least in part, on one or more signal characteristics of the data; and   determining a gesture from the data from the sensor based, at least in part, on applying the subset of gesture recognition algorithms to the data.   
     
     
         23 . The method of  claim 22 , wherein the one or more signal characteristic(s) of the data comprise at least one of
 an energy magnitude of the data,   a time duration of the data, and   a frequency spectrum of the data.   
     
     
         24 . The method of  claim 23 , wherein determining the subset of gesture recognition algorithms is based, at least in part, on at least one of
 comparing the energy magnitude of the data with one or more magnitude values associated with one of the plurality of gesture algorithms if the signal characteristic(s) of the data comprise an energy magnitude of the data,   comparing the time duration of the data with time values associated with one of the plurality of gesture algorithms if the signal characteristic(s) of the data comprise a time duration of the data, and   comparing the frequency spectrum of the data with spectrum patterns associated with one of the plurality of gesture algorithms if the signal characteristic(s) of the data comprise a frequency spectrum of the data.