Touch gesture detection on a surface with movable artifacts

Abstract

Systems and methods for touch gesture detection between two objects wherein a potential touch gesture that was detected by visual analysis of captured images of the two objects is confirmed by a correlated movement of one or more movable surface artifacts on at least one of the objects. A non-confirmed potential touch gesture may be determined to be a hovering gesture. Sound sensors that independently detect and locate finger touches, as well as motion sensors that detect vibration corresponding to actual touch tap events, may be combined with the captured images to improve the detection accuracy. For wearable smart devices like smartwatches, the skin area over the back hand with its movable surface artifacts may serve as a virtual touchpad, accepting touch gestures by the user's fingers.

1 . A computer-implemented method for confirming a touch gesture between two objects, the method comprising:
 capturing a sequence of images of said objects;   extracting from said captured images a sequence of positions of said objects;   detecting a potential touch gesture between said objects based on the proximity of their extracted positions;   extracting from said captured images, when a potential touch gesture is detected, a sequence of the shapes and the positions of one or more movable visible surface artifacts in the vicinity of the position of said detected potential touch gesture on at least one of said objects; and   confirming said potential touch gesture when change of positions or shapes of said artifacts is detected in correlation with said potential touch gesture.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein said touch gesture is a stationary touch gesture. 
     
     
         3 . The computer-implemented method of  claim 1 , wherein said touch gesture is a dragging touch gesture. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein said potential touch gesture is determined to be a hovering gesture when said extracted shapes or positions of said artifacts are not changing in correlation with said potential touch gesture during the time that said potential touch gesture is detected. 
     
     
         5 . The computer-implemented method of  claim 4 , further comprising recognizing complex gestures of one object over the other object, said complex gestures comprising intermingled sequences of touch gestures and hovering gestures. 
     
     
         6 . The computer-implemented method of  claim 5 , wherein said complex gestures are text writing gestures of one object over the other object. 
     
     
         7 . The computer-implemented method of  claim 1 , wherein one of said objects with movable surface artifacts is the skin surface over an assigned body area of a user and wherein the other object is a finger of said user. 
     
     
         8 . The computer-implemented method of  claim 7 , wherein said one or more moveable surface artifacts are selected from the group consisting of blemishes, blood vessels, wrinkles, hair, stains, or any other visible textured section of said skin surface. 
     
     
         9 . A system configured for detecting a touch gesture between two objects, comprising:
 one or more processors;   one or more cameras coupled to said one or more processors and configured to capture images of said objects;   memory; and   one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for:   extracting from said captured images a sequence of positions of said objects;   detecting a potential touch gesture between said objects based on the proximity of their extracted positions;   extracting from said captured images, when a potential touch gesture is detected, a sequence of the shapes and the positions of one or more movable visible surface artifacts in the vicinity of the position of said detected potential touch gesture on at least one of said objects; and   confirming said potential touch gesture when change of positions or shapes of said artifacts is detected in correlation with said potential touch gesture.   
     
     
         10 . The system of  claim 9 , wherein said potential touch gesture is determined to be a hovering gesture when said extracted shapes or positions of said artifacts are not changing in correlation with said potential touch gesture during the time that said potential touch gesture is detected. 
     
     
         11 . The system of  claim 10 , wherein said one or more programs further comprising instructions for recognizing complex gestures of one object over the other object, said complex gestures comprising intermingled sequences of touch gestures and hovering gestures. 
     
     
         12 . The system of  claim 9 , wherein one of said objects is the skin surface over an assigned body area of a user, and wherein said other object is a finger of said user. 
     
     
         13 . The system of  claim 12 , wherein said system is a smart watch. 
     
     
         14 . The system of  claim 13 , wherein said one or more cameras are placed on the smart phone's side facing said skin surface, and wherein each camera further comprising an optical system enhancing side viewing capabilities to enable image capturing of said finger touching said skin surface. 
     
     
         15 . The system of  claim 9 , further comprising one or more sound sensors coupled to said one or more processors, and wherein said one or more programs further comprising instructions for:
 discovering a touch gesture using said one or more sound sensors; and   confirming said discovered touch gesture based on the correlation with said potential touch gesture detected by said cameras.   
     
     
         16 . The system of  claim 9 , further comprising one or more motion sensors attached to one or more of said objects and coupled to said one or more processors, and wherein said one or more programs further comprising instructions for:
 discovering a touch gesture using said one or more motion sensors; and   confirming said discovered touch gesture based on the correlation with said potential touch gesture detected by said cameras.   
     
     
         17 . One or more non-transitory computer-readable media encoded with computer program instructions that when executed by one or more processors, cause the one or more processors to confirm a touch gesture among two or more objects by performing operations comprising:
 capturing a sequence of images of said objects;   extracting from said captured images a sequence of positions of said objects;   detecting a potential touch gesture between a pair of said objects based on the proximity of their extracted positions;   extracting from said captured images, when a potential touch gesture is detected, a sequence of the shapes and the positions of one or more movable visible surface artifacts in the vicinity of the position of said detected potential touch gesture on at least one of said objects; and   confirming said potential touch gesture when change of positions or shapes of said artifacts on at least one of said pair of objects is detected in correlation with said potential touch gesture.   
     
     
         18 . The one or more non-transitory computer-readable media of  claim 17 , wherein one of said objects is the skin surface over an assigned body area of a user, and wherein said one or more other objects are the fingers of said user. 
     
     
         19 . The one or more non-transitory computer-readable media of  claim 18 , wherein said touch gesture is a multi-finger touch gesture over said skin surface. 
     
     
         20 . The one or more non-transitory computer-readable media of  claim 18 , wherein said potential touch gesture is determined to be a hovering gesture when said extracted shapes or positions of said artifacts are not changing in correlation with said potential touch gesture during the time that said potential touch gesture is detected. 
     
     
         21 . The one or more non-transitory computer-readable media of  claim 20 , further comprising recognizing complex gestures of one object over the other object, said complex gestures comprising intermingled sequences of touch gestures and hovering gestures.