Interactive Display System

Abstract

An interactive display system including a wireless pointing device including a camera or other video capture system. The pointing device captures images displayed by the computer, including one or more human-imperceptible positioning targets. The positioning targets are presented as patterned modulation of the intensity (e.g., variation in pixel intensity) in a display frame of the visual payload, followed by the opposite modulation in a successive frame. At least two captured image frames are subtracted from one another to recover the positioning target in the captured visual data and to remove the displayed image payload. The location, size, and orientation of the recovered positioning target identify the aiming point of the remote pointing device relative to the display. Another embodiment uses temporal sequencing of positioning targets (either human-perceptible or human-imperceptible) to position the pointing device.

1 . A method of interactively and remotely operating a computer system, comprising:
 generating visual payload image frame data;   combining, with the visual payload image frame data, at least one positioning target pattern corresponding to complementary intensity variances at one or more selected pixel locations of the visual payload image frame data, applied in successive frames;   displaying the combined visual payload image frame data and positioning target pattern on a display;   capturing image data representative of at least a portion of the display including the positioning target, over at least two frames;   processing image data to subtract frames of the captured image data from one another to recover the positioning target pattern as viewed remotely from the display; and   determining a pointed-to location of the display responsive to the recovered positioning target pattern.   
     
     
         2 . The method of  claim 1 , further comprising:
 selecting a location of the display at which the positioning target pattern is to be displayed.   
     
     
         3 . The method of  claim 1 , wherein the payload image frame data corresponds to graphics display output of a computer game:
 and further comprising:
 responsive to the determining of the pointed-to location, interpreting a game command from a user; and 
 generating new visual payload image frame data responsive to the game command. 
   
     
     
         4 . The method of  claim 1 , further comprising:
 responsive to the determining of the pointed-to location, generating new visual payload image frame data;   repeating the generating, combining, displaying, capturing, processing, determining steps; and   receiving a user input from the pointing device corresponding to a record command from a user;   responsive to the record command, storing, in a memory resource of the computer system, data corresponding to a sequence of user actions including at least a plurality of pointed-to locations and corresponding user inputs.   
     
     
         5 . An interactive display system, comprising:
 a computer for generating display image data to be displayed on a display;   target generating circuitry for generating at least one positioning target pattern, the at least one positioning target pattern corresponding to complementary intensity variances to be applied to a portion of the visual payload image data in successive frames;   graphics output circuitry for combining the visual payload data with the at least one positioning target pattern into graphics output signals in a format suitable for display;   a pointing device, comprising:
 a hand-held housing; 
 a camera disposed in the housing; and 
 video capture circuitry for capturing image data obtained by the camera; and 
   positioning circuitry for subtracting successive frames captured by the pointing device from a display coupled to the computer to recover image data including the target pattern, and for determining a location of the display at which the pointing device is aimed.   
     
     
         6 . The system of  claim 5 , wherein the computer comprises:
 processing circuitry for generating the payload image data;   at least a portion of the positioning circuitry; and   wireless receiver circuitry for receiving signals from the pointing device;   and wherein the pointing device comprises:   wireless transmitter circuitry for transmitting signals corresponding to the captured image data to the computer.   
     
     
         7 . The system of  claim 5 , wherein the pointing device further comprises at least a portion of the positioning circuitry. 
     
     
         8 . The system of  claim 5 , wherein the pointing device further comprises:
 at least one actuator, for receiving user inputs.   
     
     
         9 . The system of  claim 8 , wherein the actuator is of a type selected from the group consisting of a switch, an accelerometer, a magnetic sensor, a gyroscope. 
     
     
         10 . The system of  claim 5 , wherein the pointing device further comprises:
 at least one inertial sensor.   
     
     
         11 . The system of  claim 5 , wherein the computer comprises processing circuitry for generating visual payload image data arranged in a plurality of frames;
 and wherein the graphics output circuitry combines the visual payload image frame data with the at least one positioning target pattern corresponding to complementary intensity variances at one or more selected pixel locations of the visual payload image frame data, applied in successive frames.