Visual target tracking using model fitting and exemplar

Abstract

A method of tracking a target includes receiving an observed depth image of the target from a source and analyzing the observed depth image with a prior-trained collection of known poses to find an exemplar pose that represents an observed pose of the target. The method further includes rasterizing a model of the target into a synthesized depth image having a rasterized pose and adjusting the rasterized pose of the model into a model-fitting pose based, at least in part, on differences between the observed depth image and the synthesized depth image. Either the exemplar pose or the model-fitting pose is then selected to represent the target.

1 . A method of tracking a human target, the method comprising:
 receiving an observed depth image of a scene from a source;   body scanning the scene to identify a human target in the scene;   removing non-target background information from the observed depth image;   applying to the observed depth image one or more decision trees trained from a collection of known poses to find an exemplar pose that represents an observed pose of the human target;   rasterizing a body model of the human target into a synthesized depth image having a rasterized pose;   adjusting the rasterized pose of the body model into a model-fitting pose based, at least in part, on differences between the observed depth image and the synthesized depth image;   comparing a confidence in the exemplar pose to a confidence in the model-fitting pose;   selecting the exemplar pose if the confidence in the exemplar pose is higher than or equal to the confidence in the model-fitting pose; and   selecting the model-fitting pose if the confidence in the model-fitting pose is higher than the confidence in the exemplar pose.   
     
     
         2 . The method of  claim 1 , where a terminal node of a decision tree yields a best-guess of a body part for a pixel and a confidence that the best-guess is correct. 
     
     
         3 . The method of  claim 1 , where removing non-target background information from the observed depth image includes removing depth image information outside of a sphere surrounding the target. 
     
     
         4 . The method of  claim 1 , further comprising analyzing the observed depth image with a hand-identifying algorithm configured to identify hands on the human target; and
 increasing relative confidence of the exemplar pose if the exemplar pose more closely places hands in a same location as the hand-identifying algorithm; and   increasing relative confidence of the model-fitting pose if the model-fitting pose more closely places hands in a same location as the hand-identifying algorithm.   
     
     
         5 . A method of tracking a target, the method comprising:
 receiving an observed depth image of a target from a source;   analyzing the observed depth image with a prior-trained collection of known poses to find an exemplar pose that represents an observed pose of the target;   rasterizing a model of the target into a synthesized depth image having a rasterized pose;   adjusting the rasterized pose of the model into a model-fitting pose based, at least in part, on differences between the observed depth image and the synthesized depth image; and   selecting the exemplar pose or the model-fitting pose.   
     
     
         6 . The method of  claim 5 , further comprising analyzing the observed depth image with a hand-identifying algorithm configured to identify hands on the target; and
 biasing selection of the exemplar pose or the model-fitting pose toward a pose that more closely places hands in a same location as the hand-identifying algorithm.   
     
     
         7 . The method of  claim 5 , further comprising body scanning a scene of the observed depth image to identify the target. 
     
     
         8 . The method of  claim 5 , further comprising removing non-target background information from the observed depth image. 
     
     
         9 . The method of  claim 8 , where removing non-target background information from the observed depth image includes removing depth image information outside of a three-dimensional buffer surrounding the target. 
     
     
         10 . The method of  claim 8 , where removing non-target background information from the observed depth image includes removing depth image information outside of a sphere surrounding the target. 
     
     
         11 . The method of  claim 5 , where the source includes a depth camera. 
     
     
         12 . The method of  claim 5 , where the source includes stereo cameras. 
     
     
         13 . The method of  claim 5 , where analyzing the observed depth image with a prior-trained collection of known poses includes applying to the observed depth image one or more decision trees trained from the prior-trained collection of known poses. 
     
     
         14 . The method of  claim 13 , where a terminal node of a decision tree yields a best-guess of a body part for a pixel and a confidence that the best-guess is correct. 
     
     
         15 . The method of  claim 14 , further comprising locating each joint position of the exemplar pose based, at least in part, on the best-guess of the body part for each pixel. 
     
     
         16 . The method of  claim 15 , further comprising assigning a confidence to each joint position based, at least in part, on individual confidences for each pixel. 
     
     
         17 . The method of  claim 5 , further comprising assessing a confidence of the model-fitting pose based on a comparison of the model-fitting pose and the observed pose. 
     
     
         18 . The method of  claim 5 , where selecting the exemplar pose or the model-fitting pose includes selecting the exemplar pose if a confidence in the exemplar pose is higher than or equal to a confidence in the model-fitting pose, and selecting the model-fitting pose if the confidence in the model-fitting pose is higher than the confidence in the exemplar pose. 
     
     
         19 . The method of  claim 5 , where adjusting the rasterized pose of the model into the model-fitting pose includes applying one or more forces to force-receiving locations of the model and allowing the model to move responsive to such forces. 
     
     
         20 . A computing system, comprising:
 a source configured to capture depth information;   a logic subsystem operatively connected to the source; and   a data-holding subsystem holding instructions executable by the logic subsystem to:
 receive an observed depth image of a target from a source; 
 analyze the observed depth image with a prior-trained collection of known poses to find an exemplar pose that represents an observed pose of the target; 
 rasterize a model of the target into a synthesized depth image having a rasterized pose; 
 adjust the rasterized pose of the model into a model-fitting pose based, at least in part, on differences between the observed depth image and the synthesized depth image; and 
 select the exemplar pose or the model-fitting pose.