Real-time head tracking system for computer games and other applications

Abstract

A real-time computer vision system tracks the head of a computer user to implement real-time control of games or other applications. The imaging hardware includes a color camera, frame grabber, and processor. The software consists of the low-level image grabbing software and a tracking algorithm. The system tracks objects based on the color, motion and/or shape of the object in the image. A color matching function is used to compute three measures of the target's probable location based on the target color, shape and motion. The method then computes the most probable location of the target using a weighting technique. Once the system is running, a graphical user interface displays the live image from the color camera on the computer screen. The operator can then use the mouse to select a target for tracking. The system will then keep track of the moving target in the scene in real-time.

1 . A method controlling a computer game, comprising the steps of: 
 imaging a sequence of scenes including the head of a user of the computer; and    comparing visual characteristics from scene to scene center to determine movement of the user's head within the scene; and    controlling the game in accordance with the movements.    
   
   
       2 . The method of  claim 1 , wherein the visual characteristics include color, shape or location.  
   
   
       3 . The method of  claim 1 , wherein the visual characteristics include a combination of static and dynamic characteristics.  
   
   
       4 . The method of  claim 3 , further including the step of modeling of the dynamic characteristics to yield an estimate of head position.  
   
   
       5 . The method of  claim 1 , further including the step of initiating the head tracking through a graphical user interface.  
   
   
       6 . The method of  claim 5 , wherein the graphical user interface provides a bounding box displayed in the screen to assist in targeting the user's head.  
   
   
       7 . The method of  claim 2 , further enabling a match in color despite the differences arising from lighting and shadows.  
   
   
       8 . The method of  claim 2 , further enabling a match in color within a threshold of hue.  
   
   
       9 . The method of  claim 1 , wherein the step of comparing the visual characteristics includes a comparison of pixels from scene to scene.  
   
   
       10 . The method of  claim 1 , further including the step of determining if the user's head was moved outside of the scene.  
   
   
       11 . The method of  claim 1 , wherein: 
 the visual characteristic is color; and    further including the step of finding a weighted average of color to compute the loc based upon action of the user's head based upon color alone.    
   
   
       12 . The method of  claim 1 , further including the step of segmented a region defined by a predetermined closeness of color as an estimate of target shape.  
   
   
       13 . The method of  claim 1 , further including the step of continuing to track the user's head when moving in front of or behind a similarly colored object in the scene.