Recognizing hand poses and/or object classes

Abstract

There is a need to provide simple, accurate, fast and computationally inexpensive methods of object and hand pose recognition for many applications. For example, to enable a user to make use of his or her hands to drive an application either displayed on a tablet screen or projected onto a table top. There is also a need to be able to discriminate accurately between events when a user's hand or digit touches such a display from events when a user's hand or digit hovers just above that display. A random decision forest is trained to enable recognition of hand poses and objects and optionally also whether those hand poses are touching or not touching a display surface. The random decision forest uses image features such as appearance, shape and optionally stereo image features. In some cases, the training process is cost aware. The resulting recognition system is operable in real-time.

1 . One or more computer-readable storage devices to store processor executable instructions that, when the instructions are implemented by one or more processors, configure the one or more processors to implement a method comprising:
 receiving at least one image of an item to be classified as one of a plurality of specified classes;   accessing a plurality of decision trees which have been configured by:
 a training process using information about classification accuracy; and 
 placing relatively higher cost tests called by the decision trees below relatively low tests in the decision trees, such that the relatively higher cost tests are more likely to be performed after the relatively lower cost tests; 
 classifying the image into one of the classes at least by applying the plurality of decision trees to at least part of the image using the one or more processors; and 
 storing the classified image in memory. 
   
     
     
         2 . The one or more computer-readable storage devices as claimed in  claim 1  which further comprises segmenting the at least one image to identify a foreground region and applying the plurality of decision trees to that foreground region. 
     
     
         3 . The one or more computer-readable storage devices as claimed in  claim 1  wherein the process of receiving at least one image comprises receiving a stereo image pair. 
     
     
         4 . The one or more computer-readable storage devices as claimed in  claim 1  wherein the process of receiving at least one image comprises receiving a depth map. 
     
     
         5 . The one or more computer-readable storage devices as claimed in  claim 1  which further comprises receiving a touch map and wherein the process of classifying the image comprises using information from the touch map. 
     
     
         6 . The one or more computer-readable storage devices as claimed in  claim 1  wherein the plurality of specified classes comprises classes in which items are touching a display surface and classes in which items are not touching a display surface. 
     
     
         7 . The one or more computer-readable storage devices as claimed in  claim 1  wherein the plurality of specified classes comprise a plurality of classes of hand poses and a plurality of classes of objects. 
     
     
         8 . The one or more computer-readable storage devices as claimed in  claim 1  which further comprises inputting information about the classified image into a user interface. 
     
     
         9 . The one or more computer-readable storage devices as claimed in  claim 1  wherein accessing the decision trees comprises accessing tests using any of appearance and shape image features. 
     
     
         10 . The one or more computer-readable storage devices as claimed in  claim 3  wherein accessing the decision trees comprises accessing tests using appearance image features, shape image features and stereo image features. 
     
     
         11 . The one or more computer-readable storage devices as claimed in  claim 4  wherein accessing the decision trees comprises accessing tests using depth map features. 
     
     
         12 . The one or more computer-readable storage devices as claimed in  claim 1  wherein classifying the image comprises, for each decision tree, computing a histogram using results of applying that decision tree to at least part of the image. 
     
     
         13 . The one or more computer-readable storage devices as claimed in  claim 12  wherein classifying the image further comprises, concatenating the histograms and inputting the concatenated histogram to a multi-class classifier. 
     
     
         14 . A system comprising:
 memory to store at least one image of an item to be classified as one of a plurality of specified classes;   a processor to access a plurality of decision trees which have been configured by:
 a training process using information about classification accuracy; and 
 placing relatively higher cost tests called by the decision trees below relatively low tests in the decision trees, such that the relatively higher cost tests are more likely to be performed after the relatively lower cost tests; 
 classifying the image into one of the classes at least by applying the plurality of decision trees to at least part of the image using the one or more processors. 
   
     
     
         15 . The system of  claim 14 , wherein the processor is further configured to segment the at least one image to identify a foreground region and applying the plurality of decision trees to that foreground region. 
     
     
         16 . The system of  claim 14 , wherein the plurality of specified classes comprises classes in which items are touching a display surface and classes in which items are not touching a display surface. 
     
     
         17 . The system of  claim 14 , wherein the plurality of specified classes comprise a plurality of classes of hand poses and a plurality of classes of objects. 
     
     
         18 . The system of  claim 14 , wherein classifying the image comprises, for each decision tree, computing a histogram using results of applying that decision tree to at least part of the image. 
     
     
         19 . The system of  claim 18 , wherein classifying the image further comprises, concatenating the histograms and inputting the concatenated histogram to a multi-class classifier. 
     
     
         20 . A computer implemented method comprising:
 receiving at least one image of an item to be classified as one of a plurality of specified classes in memory;   accessing a plurality of decision trees stored in memory which have been configured by:
 a training process using information about classification accuracy; and 
 placing relatively higher cost tests called by the decision trees below relatively low tests in the decision trees, such that the relatively higher cost tests are more likely to be performed after the relatively lower cost tests; 
 classifying the image into one of the classes at least by applying the plurality of decision trees to at least part of the image using the one or more processors; and 
   storing the classified image in memory.