Detecting and tracking touch on an illuminated surface using a mean-subtracted image

Abstract

A method for touch detection performed by a touch processor in an optical touch detection system is provided that includes receiving an image of an illuminated surface included in the optical touch detection system, wherein the image is captured by a camera included in the optical touch detection system, subtracting a background model from the image to generate a mean-subtracted image, identifying a set of candidate touch locations in the mean-subtracted image, classifying the candidate touch locations in the set of candidate touch locations to generate a set of validated candidate touch locations, and outputting a set of final touch locations determined from the set of validated candidate touch locations.

What is claimed is: 
     
       1. A method for touch detection performed by a touch processor in an optical touch detection system, the method comprising:
 receiving, with one or more processors, an image of an illuminated surface included in the optical touch detection system, wherein the image is captured by a camera included in the optical touch detection system; 
 subtracting, with the one or more processors, a background model from the image to generate a mean-subtracted image; 
 filtering the mean-subtracted image with a filter having zero mean with coefficients of a same sign in a central region of the filter surrounded by coefficients of an opposite sign such that a size of the central region corresponds to an expected size of a finger touch; 
 identifying, with the one or more processors, local extrema in the filtered mean-subtracted image to generate a set of candidate touch locations; 
 classifying, with the one or more processors, the candidate touch locations by comparing each of the local extrema to one or more detection thresholds to generate a set of validated candidate touch locations; and 
 outputting, with the one or more processors, a set of final touch locations determined from the set of validated candidate touch locations. 
 
     
     
       2. The method of  claim 1 , further comprising:
 identifying highly saturated pixels in the image; and 
 pruning the set of validated candidate touch locations to generate the set of final touch locations, wherein pruning the set of validated candidate touch locations comprises pruning the set of validated candidate touch locations based on the identified highly saturated pixels, wherein a validated candidate touch location is not included in the set of final touch locations if the validated candidate touch location corresponds to a highly saturated pixel. 
 
     
     
       3. The method of  claim 2 , wherein identifying highly saturated pixels comprises comparing each pixel in the image to a predetermined high saturation threshold and identifying pixels above the high saturation threshold as highly saturated pixels. 
     
     
       4. The method of  claim 2 , wherein pruning the set of validated candidate touch locations further comprises:
 removing a validated candidate touch location from the set of validated candidate touch locations when the validated candidate touch location is not in a set of foreground pixels in the image; and 
 removing a validated candidate touch location from the set of validated candidate touch locations when the validated candidate touch location is outside a maximum distance from a boundary of a blob formed by the foreground pixels. 
 
     
     
       5. The method of  claim 1 , wherein classifying the candidate touch locations comprises using a machine learning classifier to classify each candidate touch location as valid or invalid, wherein the machine learning classifier is trained to classify a candidate touch location based on a combination of features of the candidate touch location. 
     
     
       6. The method of  claim 5 , wherein the combination of features comprises at least one feature selected from a group of features consisting of a filter response value of the candidate touch location, a size of a touch region corresponding the candidate touch location, a shape of the touch region corresponding of the candidate touch location, and a texture of a neighborhood of pixels around the candidate touch location. 
     
     
       7. The method of  claim 5 , wherein the combination of features consists of a filter response value of the candidate touch location, a size of a touch region corresponding the candidate touch location, a shape of the touch region corresponding of the candidate touch location, and a texture of a neighborhood of pixels around the candidate touch location. 
     
     
       8. The method of  claim 1 , wherein a detection threshold for a first extrema pixel location is different from a detection threshold for a second extrema pixel location. 
     
     
       9. The method of  claim 1 , wherein the camera is an infrared camera. 
     
     
       10. The method of  claim 1 , further comprising:
 tracking the final touch locations, wherein tracking comprises:
 using Kalman filter prediction to compute an expected location of each touch location from a previous image; and 
 using Munkres' assignment algorithm to determine which touch locations of the final touch locations correspond to touch locations from the previous image. 
 
 
     
     
       11. An optical touch detection system configured for touch detection, the system comprising:
 an illuminated surface; 
 a camera positioned to capture images of the illuminated surface; and 
 one or more processors configured to:
 receive an image of the illuminated surface captured by the camera; 
 subtract a background model from the image to generate a mean-subtracted image; 
 identify local extrema in the mean-subtracted image to generate a set of candidate touch locations; 
 classify the candidate touch locations in the set of candidate touch locations by comparing each of the local extrema to one or more detection thresholds to generate a set of validated candidate touch locations; and 
 output a set of final touch locations determined from the set of validated candidate touch locations. 
 
 
     
     
       12. The optical touch detection system of  claim 11 , wherein the one or more processors are further configured to:
 identify highly saturated pixels in the image; and 
 prune the set of validated candidate touch locations, based on the identified highly saturated pixels, to generate the set of final touch locations, wherein a validated candidate touch location is not included in the set of final touch locations if the validated candidate touch location corresponds to a highly saturated pixel. 
 
     
     
       13. The optical touch detection system of  claim 11 , wherein the one or more processors are further configured to use a machine learning classifier to classify each candidate touch location as valid or invalid, wherein the machine learning classifier is trained to classify a candidate touch location based on a combination of features of the candidate touch location. 
     
     
       14. The optical touch detection system of  claim 11 , wherein the combination of features comprises at least one feature selected from a group of features consisting of a filter response value of the candidate touch location, a size of a touch region corresponding the candidate touch location, a shape of the touch region corresponding of the candidate touch location, and a texture of a neighborhood of pixels around the candidate touch location. 
     
     
       15. The optical touch detection system of  claim 11 , wherein the one or more processors are further configured to: compare each pixel to a predetermined detection threshold, the predetermined pixel threshold corresponding to a location of the pixel, wherein a predetermined detection threshold for a first pixel location is different from a predetermined detection threshold for a second pixel location. 
     
     
       16. The optical touch detection system of  claim 11 , wherein the one or more processors are further configured to:
 filter the mean-subtracted image with a filter having zero mean with coefficients of a same sign in a center of the filter surrounded by coefficients of an opposite sign such that a size of a central region corresponds to an expected size of a finger touch; and 
 identify local extrema in the filtered mean-subtracted image. 
 
     
     
       17. The optical touch detection system of  claim 11 , wherein the camera is an infrared camera. 
     
     
       18. A non-transitory computer readable medium storing software instructions that, when executed by a touch processor included in an optical touch detection system, cause the optical touch detection system to: receive an image of an illuminated surface included in the optical touch detection system, wherein the image is captured by a camera included in the optical touch detection system; subtract a background model from the image to generate a mean-subtracted image; identify local extrema in the mean-subtracted image to generate a set of candidate touch locations; classify the candidate touch locations in the set of candidate touch locations by comparing each of the local extrema to one or more detection thresholds to generate a set of validated candidate touch locations; and output a set of final touch locations determined from the set of validated candidate touch locations.