Gesture Based Interface System and Method

Abstract

A user interface apparatus for controlling any kind of a device. Images obtained by an image sensor in a region adjacent to the device are input to a gesture recognition system which analyzes images obtained by the image sensor to identify one or more gestures. A message decision maker generates a message based upon an identified gesture and a recognition mode of the gesture recognition system. The recognition mode is changed under one or more various conditions.

1 .- 67 . (canceled) 
     
     
         68 . An apparatus, comprising:
 at least one processor; and   a memory storing instructions, wherein the at least one processor is configured to execute the stored instructions to perform operations including:
 receiving image data from one or more image sensors associated with a device; 
 enabling processing of image data from the one or more image sensors in at least first and second modes, wherein at least the second mode is associated with a second set of user information relating to user behavior; 
 detecting a posture of a body part, a shape of the body part, a position of the body part, or an orientation of the body part; 
 changing a recognition mode of the device from a first recognition mode to a second recognition mode based on the detection; and 
 generating a message associated with the change to the second recognition mode to indicate detection of the second set of user information relating to user behavior. 
   
     
     
         69 . The apparatus of  claim 68 , wherein
 the at least one processor is further configured to detect a combined gesture, wherein the combined gesture comprises the posture of the body part over one or more instances in an interval, and   the at least one processor changes the recognition mode of the device based at least in part on detection of the combined gesture.   
     
     
         70 . The apparatus of  claim 68 , wherein
 the device is a vehicle,   the at least one processor is further configured to obtain information related to the posture of the body part and information related to a movement of the vehicle, and   change the recognition mode of the device from the first recognition mode to the second recognition mode based at least in part on the information related to the posture of the body part and the information related to the movement of the vehicle.   
     
     
         71 . The apparatus of  claim 68 , wherein the at least one processor is further configured to:
 obtain information related to the posture of the body part and information related to one or more triggers, and   change the recognition mode of the device from the first recognition mode to the second recognition mode based at least in part on the information related to the posture of the body part and the information related to one or more triggers.   
     
     
         72 . The apparatus of  claim 68 , wherein the posture of the body part is detected based at least in part on two or more of the shape of the body part, the body part position, and the body part orientation. 
     
     
         73 . The apparatus of  claim 68 , wherein the at least one body part is selected from a face, one or two eyes, a hand, a portion of a hand, or a pose of a hand. 
     
     
         74 . The apparatus of  claim 68 , wherein the second recognition mode remains active for a predefined period of time after the detection. 
     
     
         75 . The apparatus of  claim 68 , wherein an amount of information received in the first mode is greater than an amount of information received in the second mode. 
     
     
         76 . The apparatus of  claim 75 , wherein the amount of information is associated with a frame rate or image resolution of the received image data. 
     
     
         77 . The apparatus of  claim 68 , wherein the processor is further configured to:
 receive a sequence of depth maps over a predetermined period of time of the body part; and   detect, based at least in part on the received sequence of depth maps, a direction or speed of movement of the at least one body part.   
     
     
         78 . The apparatus of  claim 68 , wherein the generated message is addressed to the device, to an operating system of the device, or to one or more applications running on a processor of the device. 
     
     
         79 . The apparatus of  claim 68 , wherein the processor is further configured to:
 detect a first body part and a second body part;   compare a relative size or a spatial relationship between the first body part and the second body part; and   change the recognition mode of the device based at least in part on the comparison of the first and second body parts.   
     
     
         80 . A computer-implemented method for controlling a device, the method being performed by at least one processor and comprising:
 receiving image data from one or more image sensors associated with a device;   enabling processing of image data from the one or more image sensors in at least first and second modes, wherein at least the second mode is associated with a second set of user information relating to user behavior;   detecting a posture of a body part, a shape of the body part, a position of the body part, or an orientation of the body part;   changing a recognition mode of the device from a first recognition mode to a second recognition mode based at least in part on the detection; and   generating a message associated with the change to the second recognition mode to indicate detection of second set of user information relating to user behavior.   
     
     
         81 . The computer-implemented method of  claim 80 , wherein the device is a vehicle, and the processor is further configured to:
 obtain information related to the posture of the body part and information related to a movement of the vehicle, and   change the recognition mode of the device from the first recognition mode to the second recognition mode based at least in part on the information related to the posture of the body part and the information related to the movement of the vehicle.   
     
     
         82 . The computer-implemented method of  claim 80 , wherein the processor is further configured to:
 obtain information related to the posture of the body part and information related to one or more triggers, and   change the recognition mode of the device from the first recognition mode to the second recognition mode based at least in part on the information related to the posture of the body part and the information related to one or more triggers.   
     
     
         83 . The computer-implemented method of  claim 80 , wherein the processor is further configured to detect a combined gesture, wherein the combined gesture comprises the posture of the at least one body part over one or more instances in an interval. 
     
     
         84 . The computer-implemented method of  claim 83 , wherein the processor is further configured to change the recognition mode of the device based at least in part on the detection of the combined gesture. 
     
     
         85 . The computer-implemented method of  claim 80 , wherein the at least one body part is selected from a face, one or two eyes, a hand, a portion of a hand, or a pose of a hand. 
     
     
         86 . The computer-implemented method of  claim 80 , wherein, after the detection of the posture, the second recognition mode remains active for a predefined period of time corresponding to user behavioral patterns. 
     
     
         87 . The computer-implemented method of  claim 80 , wherein an amount of information received in the first mode is greater than an amount of information received in the second mode. 
     
     
         88 . The computer-implemented method of  claim 87 , wherein a frame rate of the image sensor is less than 8 frames per second in the first mode. 
     
     
         89 . The computer-implemented method of  claim 80 , wherein the processor is further configured to:
 capture a sequence of depth maps over a predetermined period of time of the at least one body part; and   detect a direction or speed of movement of the at least one body part.   
     
     
         90 . The computer-implemented method of  claim 80 , wherein the generated message is addressed to the device, to an operating system of the device, or to one or more applications running on a processor of the device. 
     
     
         91 . The computer-implemented method of  claim 80 , wherein the processor is further configured to:
 detect a first body part and a second body part;   compare relative size or spatial relationship between the first body part and the second body part; and   adjust the recognition mode of the device based at least in part on the comparison of the first and second body parts.