Detecting User Input Provided To A Projected User Interface

Abstract

Apparatus and methods for determining whether a user intends to provide an input using an image of a control appearing on a surface. An apparatus may include a first camera to capture two or more images of the surface, and a unit to determine whether various conditions are true. A first condition is that a particular number of pixels classified as skin color are present within one cell of the two or more images. A cell has a location substantially coinciding with the image of the control. A second condition is that the pixels classified as skin color persist for at least a particular time period. A third condition is that the pixels classified as skin color have a first shape. Additional conditions are disclosed. A signal indicative of an intent of a user to provide an input may be provided if the each of the first, second, and third conditions are true.

1 . An apparatus for determining whether a user intends to provide an input using an image of a control appearing on a surface, comprising:
 a first camera to capture two or more images of the surface; and   a unit to determine whether first, second, and third conditions are true, the first condition being that a particular number of pixels classified as skin color are present within one cell of the two or more images, the cell having a location substantially coinciding with the image of the control, the second condition being that the pixels classified as skin color persist for at least a particular time period, the third condition being that the pixels classified as skin color have a first shape, and to provide a signal indicative of an intent of a user to provide an input if the each of the first, second, and third conditions are true.   
     
     
         2 . The apparatus of  claim 1 , further comprising a projector to project one or more user controls onto the surface. 
     
     
         3 . The apparatus of  claim 1 , wherein the camera and the surface are in a fixed spatial relationship with one another. 
     
     
         4 . The apparatus of  claim 1 , further comprising:
 a second camera to capture two or more images of the surface, the second camera being spaced apart from the first camera; and wherein   the unit determines whether a fourth condition is true, the fourth condition being that the pixels classified as skin color and having the first shape are within a first distance from the surface, and the unit provides a signal indicative of an intent of a user to provide an input if the each of the first, second, third and fourth conditions are true.   
     
     
         5 . The apparatus of  claim 4 , wherein the unit provides a signal indicative of an intent of a user to provide an input if a majority of the first, second, third, and fourth conditions are true. 
     
     
         6 . The apparatus of  claim 4 , wherein the unit determines whether a fifth condition is true, the fifth condition being that a particular number of pixels classified as finger-nail color are present within the cell of the two or more images, and that a first count of the pixels classified as finger-nail color at a first time is greater than a second count of the pixels classified as finger-nail color at a second time, and the unit provides a signal indicative of an intent of a user to provide an input if the each of the first, second, third, fourth, and fifth conditions are true. 
     
     
         7 . The apparatus of  claim 6 , wherein the unit determines whether a sixth condition is true, the sixth condition being that a first position at a first time of the pixels classified as skin color is different from a second position at a second time of the pixels classified as skin color, and the unit provides a signal indicative of an intent of a user to provide an input if the each of the first, second, third, fourth, fifth, and sixth conditions are true. 
     
     
         8 . The apparatus of  claim 7 , wherein the unit provides a signal indicative of an intent of a user to provide an input if a majority of the first, second, third, fourth, fifth, and sixth seventh conditions are true. 
     
     
         9 . A method for determining whether a user intends to provide an input using an image of a control appearing on a surface, comprising:
 capturing two or more images of the surface;   determining whether first, second, and third conditions are true, the first condition being that a particular number of pixels classified as skin color are present within one cell of the two or more images, the cell having a location substantially coinciding with the image of the control, the second condition being that the pixels classified as skin color persist for at least a particular time period, the third condition being that the pixels classified as skin color have a first shape; and   providing a signal indicative of an intent of a user to provide an input if the each of the first, second, and third conditions are true.   
     
     
         10 . The method of  claim 9 , further comprising projecting one or more user controls onto the surface. 
     
     
         11 . The method of  claim 9 , maintaining the camera and the surface in a fixed spatial relationship with one another. 
     
     
         12 . The method of  claim 9 , further comprising:
 capturing two or more images of the surface from a second perspective; and   determining whether a fourth condition is true, the fourth condition being that the pixels classified as skin color and having the first shape are within a first distance from the surface;   wherein the signal indicative of an intent of a user to provide an input is provided if the each of the first, second, third and fourth conditions are true.   
     
     
         13 . The method of  claim 12 , wherein the signal indicative of an intent of a user to provide an input is provided if a majority of the first, second, third, and fourth conditions are true. 
     
     
         14 . The method of  claim 12 , further comprising:
 determining whether a fifth condition is true, the fifth condition being that a particular number of pixels classified as finger-nail color are present within the cell of the two or more images; and   determining that a first count of the pixels classified as finger-nail color at a first time is greater than a second count of the pixels classified as finger-nail color at a second time;   wherein the signal indicative of an intent of a user to provide an input is provided if the each of the first, second, third, fourth, and fifth conditions are true.   
     
     
         15 . The method of  claim 14 , further comprising:
 determining whether a sixth condition is true, the sixth condition being that a first position at a first time of the pixels classified as skin color is different from a second position at a second time of the pixels classified as skin color;   wherein the signal indicative of an intent of a user to provide an input is provided if the each of the first, second, third, fourth, fifth, and sixth conditions are true.   
     
     
         16 . The method of  claim 15 , wherein the signal indicative of an intent of a user to provide an input is provided if a majority of the first, second, third, fourth, fifth, and sixth seventh conditions are true.