Object direction using video input combined with tilt angle information

Abstract

A method for use in obtaining input data from an object includes capturing a live image of the object with an image capture device, receiving information related to a tilt angle of the object from one or more sensors other than the image capture device, using the information related to a tilt angle of the object to obtain at least one rotated reference image of the object, comparing the live image of the object with the rotated reference image of the object, and generating an indication in response to the live image matching the rotated reference image. An apparatus and a storage medium storing a computer program executable by a processor based system are also disclosed

What is claimed is: 
     
       1. A method for use in obtaining input data from an object, comprising:
 capturing a live image of the object with an image capture device; 
 receiving information related to a tilt angle of the object from one or more sensors other than the image capture device; 
 using the information related to a tilt angle of the object to obtain at least one rotated reference image of the object; 
 comparing the live image of the object with the rotated reference image of the object; and 
 generating an indication in response to the live image matching the rotated reference image. 
 
     
     
       2. A method in accordance with  claim 1 , wherein the one or more sensors other than the image capture device comprises a gyro sensor used in the object. 
     
     
       3. A method in accordance with  claim 1 , wherein the one or more sensors other than the image capture device comprises an accelerometer used in the object. 
     
     
       4. A method in accordance with  claim 1 , wherein the rotated reference image of the object is obtained by retrieving it from storage. 
     
     
       5. A method in accordance with  claim 1 , wherein the rotated reference image of the object is obtained by rotating a reference image of the object. 
     
     
       6. A method in accordance with  claim 1 , further comprising:
 passing the live image of the object and the rotated reference image of the object through a normalizing filter. 
 
     
     
       7. A method in accordance with  claim 1 , wherein the image capture device comprises functions as a camera. 
     
     
       8. A method in accordance with  claim 1 , wherein the image capture device comprises functions as a Z-camera. 
     
     
       9. A method in accordance with  claim 1 , wherein the object comprises functions as a hand-held controller. 
     
     
       10. A method in accordance with  claim 1 , wherein the information relating to the tilt angle of the object is used to obtain a plurality of rotated reference images of the object, and the live image of the object is compared with each of the plurality of rotated reference images of the object. 
     
     
       11. An apparatus for use in obtaining input data from an object, comprising:
 an image capture device configured to capture a live image of the object; and 
 a processor based system configured to receive information related to a tilt angle of the object from one or more sensors other than the image capture device, use the information related to a tilt angle of the object to obtain at least one rotated reference image of the object, compare the live image of the object with the rotated reference image of the object, and generate an indication in response to the live image matching the rotated reference image. 
 
     
     
       12. An apparatus in accordance with  claim 11 , wherein the one or more sensors other than the image capture device comprises a gyro sensor in the object. 
     
     
       13. An apparatus in accordance with  claim 11 , wherein the one or more sensors other than the image capture device comprises an accelerometer in the object. 
     
     
       14. An apparatus in accordance with  claim 11 , wherein the rotated reference image of the object is obtained by retrieving it from storage. 
     
     
       15. An apparatus in accordance with  claim 11 , wherein the rotated reference image of the object is obtained by rotating a reference image of the object. 
     
     
       16. An apparatus in accordance with  claim 11 , wherein the processor based system is further configured to pass the live image of the object and the rotated reference image of the object through a normalizing filter. 
     
     
       17. An apparatus in accordance with  claim 11 , wherein the image capture device comprises a camera. 
     
     
       18. An apparatus in accordance with  claim 11 , wherein the object comprises a hand-held controller. 
     
     
       19. An apparatus in accordance with  claim 11 , wherein the information relating to the tilt angle of the object is used to obtain a plurality of rotated reference images of the object, and the live image of the object is compared with each of the plurality of rotated reference images of the object. 
     
     
       20. A non-transitory computer readable storage medium storing a computer program executable by a processor based system, the computer program causing the processor based system to execute steps comprising:
 receiving a captured live image of an object from an image capture device; 
 receiving information related to a tilt angle of the object from one or more sensors other than the image capture device; 
 using the information related to a tilt angle of the object to obtain at least one rotated reference image of the object; 
 comparing the live image of the object with the rotated reference image of the object; and 
 generating an indication in response to the live image matching the rotated reference image. 
 
     
     
       21. A storage medium in accordance with  claim 20 , wherein the one or more sensors other than the image capture device comprises a gyro sensor used in the object. 
     
     
       22. A storage medium in accordance with  claim 20 , wherein the one or more sensors other than the image capture device comprises an accelerometer used in the object. 
     
     
       23. A storage medium in accordance with  claim 20 , wherein the rotated reference image of the object is obtained by retrieving it from storage. 
     
     
       24. A storage medium in accordance with  claim 20 , wherein the rotated reference image of the object is obtained by rotating a reference image of the object. 
     
     
       25. A storage medium in accordance with  claim 20 , wherein the computer program is further configured to cause the processor based system to execute a step comprising:
 passing the live image of the object and the rotated reference image of the object through a normalizing filter. 
 
     
     
       26. A storage medium in accordance with  claim 20 , wherein the image capture device comprises functions as a camera. 
     
     
       27. A storage medium in accordance with  claim 20 , wherein the image capture device comprises functions as a Z-camera. 
     
     
       28. A storage medium in accordance with  claim 20 , wherein the object comprises functions as a hand-held controller. 
     
     
       29. A storage medium in accordance with  claim 20 , wherein the information relating to the tilt angle of the object is used to obtain a plurality of rotated reference images of the object, and the live image of the object is compared with each of the plurality of rotated reference images of the object. 
     
     
       30. A method for tracking a device having a body to be worn by a user to produce input data, comprising:
 capturing a plurality of images of the body with an image capture device, the body including infrared (IR) light emitting diodes (LEDs) and the images captured of the body including at least one or more of the IR LEDs for detecting changes in position or orientation of the body in three-dimensional space;   receiving information related to motion of the body from one or more sensors other than the image capture device while capturing said plurality of images of the body, the information related to motion of the body used to identify three-dimensional movements of the body and a tilt angle of the body;   using the images of the body that include the one or more IR LEDs to identify a first part of the tracking information of the body and using the information related to the motion of the body to identify a second part of the tracking information, the second part of the tracking information includes at least one rotated reference image; and   determining the input data based on a combination of both the first part of the tracking information and the second part of the tracking information, the combination includes comparing the at least one rotated reference image with one or more of said plurality of images of the body, wherein the input data is used to control virtual fields of view and virtual movements in a scene of a virtual world when presented on a screen.    
     
     
       31. The method of claim 30, wherein the one or more sensors other than the image capture device include one of a gyro sensor used in the body of the device, or an accelerometer used in the body of the device, or both a gyro sensor and an accelerometer used in the body of the device;
 wherein the device is configured for communication with a computer system that processes instructions for the capturing, the receiving, and the using of the method.    
     
     
       32. The method of claim 31, wherein the communication used is wireless.  
     
     
       33. The method of claim 31, wherein the image capture device is functions to capture images that include a head of the user.  
     
     
       34. The method of claim 30, wherein the image capture device is an infrared (IR) capable camera positioned in a direction that faces in a direction of the body having the IR LEDs, the body used in a space viewable by the image capture device.  
     
     
       35. The method of claim 30, wherein the one or more sensors other than the image capture device function as one or more Micro-Electro-Mechanical Systems (MEMS) sensors.  
     
     
       36. The method of claim 30, wherein detecting changes in position and orientation of the body in three-dimensional space involves analyzing the plurality of images for a pattern of the one or more IR LEDs.  
     
     
       37. The method of claim 30, wherein the determining the input data includes mixing the first part of the tracking data with the second part of the tracking data to generate refined position and orientation information for the virtual movements.  
     
     
       38. The method of claim 30, wherein determining the input data based on a combination of both the first part of the tracking information and the second part of the tracking information includes mapping movements of the body to predefined input commands.  
     
     
       39. A method for tracking a device having a housing, the device is used to provide input data, comprising:
 establishing communication between the device and a computer;   receiving, by the computer, inertial sensor data of the device from one or more sensors, the inertial sensor data producing information related to a tilt angle of the housing used to obtain at least one rotated reference image;   receiving a plurality of images of the housing of the device from an image capture device, the housing being disposed in space substantially in front of the image capture device, the housing includes infrared (IR) light emitting diodes (LEDs), wherein one or more of the plurality of images captured by the image capture device include at least one or more of the IR LEDs disposed on the housing; and   correcting, at least partially, drift in the inertial sensor data using one or more of the plurality of images captured by the image capture device;   tracking the housing using a combination of information obtained from the plurality of images and information obtained from the inertial sensor data, the combination includes comparing the at least one reference image with one or more of said plurality of images of the housing, the tracking of the device producing the input data that is used to control virtual movements and virtual views presented on a display screen.    
     
     
       40. The method of claim 39, wherein the one or more sensors other than the image capture device include one of a gyro sensor used in the housing of the device, or an accelerometer used in the housing of the device, or both a gyro sensor and an accelerometer used in the housing of the device.  
     
     
       41. The method of claim 39, wherein the housing functions to be worn or held by a user.  
     
     
       42. The method of claim 39, wherein tracking the device includes analyzing the plurality of images for a pattern of the one or more IR LEDs.  
     
     
       43. The method of claim 39, wherein producing the input data includes mapping movements of the housing to predefined input commands.