Image processor configured for efficient estimation and elimination of foreground information in images

Abstract

An image processing system comprises an image processor having image processing circuitry and an associated memory. The image processor is configured to implement a foreground processing module utilizing the image processing circuitry and the memory. The foreground processing module is configured to obtain one or more images, to estimate a foreground region of interest from the one or more images, to determine a plurality of segments of the foreground region of interest, to calculate amplitude statistics for respective ones of the plurality of segments, to classify respective segments as being respective portions of static foreground objects or as being respective portions of dynamic foreground objects based at least in part on the calculated amplitude statistics and one or more defined patterns for known static and dynamic objects, and to remove one or more segments classified as static foreground objects from the foreground region of interest.

What is claimed is: 
     
       1. A method comprising the steps of:
 obtaining one or more images; 
 estimating a foreground region of interest from the one or more images; 
 determining a plurality of segments of the foreground region of interest; 
 calculating amplitude statistics for respective ones of the plurality of segments; 
 classifying respective segments as being respective portions of static foreground objects or as being respective portions of dynamic foreground objects based at least in part on the calculated amplitude statistics and one or more defined patterns for known static and dynamic objects; and 
 removing one or more segments classified as static foreground objects from the foreground region of interest; 
 wherein the steps are implemented in an image processor comprising a processor coupled to a memory. 
 
     
     
       2. The method of  claim 1  further comprising calculating depth statistics for respective ones of the plurality of segments, wherein classifying respective segments as being respective portions of static foreground objects or as being respective portions of dynamic foreground objects is based at least in part on the calculated amplitude statistics, the calculated depth statistics and the one or more defined patterns for known static and dynamic objects. 
     
     
       3. The method of  claim 1  wherein estimating the foreground region of interest comprises generating a binary foreground region of interest mask in which pixels within the foreground region of interest all have a first binary value and pixels outside the foreground region of interest all have a second binary value complementary to the first binary value. 
     
     
       4. The method of  claim 3  wherein:
 the one or more images comprise a amplitude image and a corresponding depth image; 
 respective pixels in the amplitude image having amplitude values above a defined threshold are assigned the first binary value and respective pixels having amplitude values below the defined threshold are assigned the second binary value; and 
 respective pixels in the depth image having a depth within a defined threshold are assigned the first binary value and respective pixels having a depth outside the defined threshold are assigned the second binary value. 
 
     
     
       5. The method of  claim 1  wherein determining the plurality of segments of the foreground region of interest comprises one of:
 segmenting the foreground region of interest into a predefined constant number of segments; 
 segmenting the foreground region of interest into segments comprising respective groups of pixels such that respective numbers of pixels in each group of pixels is approximately the same; and 
 segmenting the foreground region of interest into areas defined by a rectangular grid. 
 
     
     
       6. The method of  claim 1  wherein obtaining the one or more images comprises obtaining a amplitude image, estimating per-pixel amplitude statistics for each pixel of the amplitude image, and storing the per-pixel amplitude statistics in a amplitude history buffer. 
     
     
       7. The method of  claim 6  wherein calculating amplitude statistics for respective ones of the plurality of segments is based on an analysis of the per-pixel amplitude statistics of pixels within the respective segments. 
     
     
       8. The method of  claim 7  wherein the amplitude statistics of a given segment comprise an arithmetic mean value of per-pixel amplitude values of pixels within the given segment and a standard deviation of the arithmetic mean value. 
     
     
       9. The method of  claim 1  further comprising classifying respective segments as being skin material or as being non-skin material based at least in part on the calculated amplitude statistics and one or more defined patterns for skin and non-skin materials. 
     
     
       10. The method of  claim 9  further comprising removing one or more segments classified as non-skin material from the foreground region of interest. 
     
     
       11. The method of  claim 9  further comprising classifying one or more of the plurality of segments of the foreground region of interest as a hand region of interest responsive to classifying the one or more segments as dynamic objects and classifying the one or more objects as being skin material. 
     
     
       12. The method of  claim 11  further comprising removing from the foreground region of interest segments not classified as the hand region of interest. 
     
     
       13. The method of  claim 9  further comprising:
 building a connectivity graph for the plurality of segments; 
 assigning a skin flag to each segment, wherein the skin flag identifies whether respective segments are classified as skin material or non-skin material; 
 propagating the skin flag based on the connectivity graph to define one or more skin areas of the foreground region of interest; 
 assigning a dynamic flag to each segment, wherein the dynamic flag identifies whether respective segments are classified as being respective portions of static foreground objects or as being respective portions of dynamic foreground objects; 
 propagating the dynamic flag based on the connectivity graph to define one or more dynamic areas of the foreground region of interest; and 
 defining dynamic skin areas of the foreground region of interest as intersections of the skin and dynamic areas. 
 
     
     
       14. The method of  claim 13  further comprising:
 classifying at least a portion of the dynamic skin area as a hand region of interest; and 
 removing from the foreground region of interest segments not classified as the hand region of interest. 
 
     
     
       15. An apparatus comprising:
 an image processor comprising image processing circuitry and an associated memory; 
 wherein the image processor is configured to implement a foreground processing module utilizing the image processing circuitry and the memory; 
 wherein the foreground processing module is configured:
 to obtain one or more images; 
 to estimate a foreground region of interest from the one or more images; 
 to determine a plurality of segments of the foreground region of interest; 
 to calculate amplitude statistics for respective ones of the plurality of segments; 
 to classify respective segments as being respective portions of static foreground objects or as being respective portions of dynamic foreground objects based at least in part on the calculated amplitude statistics and one or more defined patterns for known static and dynamic objects; and 
 to remove one or more segments classified as static foreground objects from the foreground region of interest. 
 
 
     
     
       16. The apparatus of  claim 15  wherein the foreground processing module is further configured:
 to classify respective segments as being skin material or as being non-skin material based at least in part on the calculated amplitude statistics and one or more defined patterns for skin and non-skin materials; and 
 to remove one or more segments classified as non-skin material from the foreground region of interest. 
 
     
     
       17. The apparatus of  claim 16  wherein the foreground processing module is further configured:
 to classify one or more of the plurality of segments of the foreground region of interest as a hand region of interest responsive to classifying the one or more segments as dynamic objects and classifying the one or more objects as being skin material; and 
 to remove from the foreground region of interest segments not classified as the hand region of interest. 
 
     
     
       18. The apparatus of  claim 15  wherein the one or more images comprise a amplitude image and a corresponding depth image. 
     
     
       19. The apparatus of  claim 18  wherein respective pixels in the amplitude image having amplitude values above a defined threshold are assigned a first binary value and respective pixels having amplitude values below the defined threshold are assigned the second binary value. 
     
     
       20. The apparatus of  claim 19  wherein respective pixels in the depth image having a depth within a defined threshold are assigned the first binary value and respective pixels having a depth outside the defined threshold are assigned a second binary value.