Sensor-based distributed tangible user interface

Abstract

A distributed tangible user interface comprises compact, self-powered, tangible user interface manipulative devices having sensing, display, and wireless communication capabilities, along with one or more associated digital content or other interactive software management applications. The manipulative devices display visual representations of digital content or program controls and can be physically manipulated as a group by a user for interaction with the digital information or software application. A controller on each manipulative device receives and processes data from a movement sensor, initiating behavior on the manipulative and/or forwarding the results to a management application that uses the information to manage the digital content, software application, and/or the manipulative devices. The manipulative devices may also detect the proximity and identity of other manipulative devices, responding to and/or forwarding that information to the management application, and may have feedback devices for presenting responsive information to the user.

1 . A tangible user interface, comprising:
 a plurality of tangible user interface manipulative devices, each tangible user interface manipulative device being independently manipulable relative to the other tangible user interface manipulative devices, each tangible user interface manipulative device comprising:
 at least one wireless communications device; 
 a visual display for digital content; 
 a power source; 
 at least one movement sensor; and 
 at least one controller adapted for:
 receiving data from the movement sensor; 
 processing the received data to derive movement parameters; and 
 at least one of forwarding the derived movement parameters or initiating tangible user interface behaviour in response to the derived movement parameters; and 
 
   at least one management application, the management application being adapted for:
 sending digital content or behavior instructions to individual tangible user interface manipulative devices; 
 receiving derived movement parameters from at least one of the tangible user interface manipulative devices; 
 processing derived movement parameters to derive instructions about management of the digital content or program behavior; and 
 changing program behavior or managing the digital content according to the derived instructions. 
   
   
   
       2 . The tangible user interface of  claim 1 , the management application being further adapted for sending at least one of revised digital content or behavior instructions to individual tangible user interface manipulative devices according to the derived instructions. 
   
   
       3 . The tangible user interface of  claim 1 , the tangible user interface manipulative devices further comprising
 at least one neighborhood wireless communications device for sensing nearby tangible user interface manipulative devices; and   the at least one controller being further adapted for:
 sensing, using the neighborhood wireless communication device, the position of at least one nearby tangible user interface manipulative device; 
 processing the position of, and any communication received from, the sensed nearby tangible user interface manipulative device in order to derive neighborhood information; and 
 at least one of forwarding the derived neighbourhood information to the management application or initiating tangible user interface behavior in response to the derived neighbourhood information. 
   
   
   
       4 . The tangible user interface of  claim 3 , the management application being further adapted for sending at least one of revised digital content or behavior instructions to individual tangible user interface manipulative devices according to the derived instructions. 
   
   
       5 . The tangible user interface of  claim 1 , the tangible user interface manipulative devices further comprising at least one feedback device for presenting responsive information to a user. 
   
   
       6 . The tangible user interface of  claim 3 , the tangible user interface manipulative devices further comprising at least one feedback device for presenting responsive information to a user. 
   
   
       7 . The tangible user interface of  claim 1 , wherein the management application resides on a tangible user interface device. 
   
   
       8 . The tangible user interface of  claim 3 , wherein the management application resides on a tangible user interface device. 
   
   
       9 . The tangible user interface of  claim 1 , wherein the movement sensor is an accelerometer. 
   
   
       10 . The tangible user interface of  claim 1 , the tangible user interface manipulative devices further comprising memory for storing at least one of digital content or program instructions. 
   
   
       11 . A tangible user interface manipulative device, comprising
 a visual display;   at least one wireless communications device, the wireless communications device being adapted for receiving behaviour commands or digital content for display on the visual display;   a power source;   at least one movement sensor; and   at least one controller adapted for:
 receiving data from the movement sensor; and 
 at least one of:
 processing the received data to derive movement parameters; 
 initiating behaviour as a result of the data or movement parameters; and 
 forwarding the derived movement parameters or the received data using the wireless communications device. 
 
   
   
   
       12 . The tangible user interface manipulative device of  claim 11 , further comprising:
 at least one neighborhood wireless communications device for sensing nearby tangible user interface manipulative devices; and   the at least one controller being further adapted for:
 sensing, using the neighborhood wireless communication device, the position of at least one nearby tangible user interface manipulative device; 
 processing the position of, and any communication received from, the sensed nearby tangible user interface manipulative device in order to derive neighborhood information; and 
 at least one of forwarding the derived neighbourhood information using the wireless communications device or initiating tangible user interface behavior in response to the derived neighbourhood information. 
   
   
   
       13 . The tangible user interface manipulative device of  claim 11 , wherein the derived movement parameters are forwarded to a computing device. 
   
   
       14 . The tangible user interface manipulative device of  claim 12 , wherein the derived movement parameters and derived neighborhood information are forwarded to a computing device. 
   
   
       15 . The tangible user interface manipulative device of  claim 11 , further comprising at least one feedback device for presenting responsive information to a user. 
   
   
       16 . The tangible user interface manipulative device of  claim 11 , wherein the movement sensor is an accelerometer. 
   
   
       17 . The tangible user interface manipulative devices of  claim 11 , further comprising memory for storing at least one of digital content or program instructions. 
   
   
       18 . A method for facilitating user interaction with digital content or application programs, comprising the steps of:
 displaying a visual representation of at least one of digital content or program control elements on a plurality of tangible user interface manipulative devices, such that a subset of the digital content or program control elements is displayed on any individual device;   detecting at least one of a manipulation of at least one of the tangible user interface manipulative devices or a location-based relationship between at least two of the tangible user interface manipulative devices; and   deriving digital content relationship information or instructions from the detected manipulation or relationship.   
   
   
       19 . The method of  claim 18 , further comprising the step of forwarding the derived digital content relationship information or instructions to a computing device. 
   
   
       20 . The method of  claim 19 , further comprising the step of managing the digital content or application program behavior according to the derived digital content relationship information or instructions received by the computing device. 
   
   
       21 . The method of  claim 20 , further comprising the step of sending at least one of revised digital content or behavior instructions to individual tangible user interface manipulative devices according to the derived digital content relationship information or instructions received by the computing device. 
   
   
       22 . The method of  claim 19 , further comprising the steps of:
 processing the derived digital content relationship information or instructions received by the computing device; and   managing the digital content or application program behavior according to the processed digital content relationship information or instructions.   
   
   
       23 . The method of  claim 22 , further comprising the step of sending revised digital content or behavior instructions to individual tangible user interface manipulative devices according to the processed digital content relationship information or instructions. 
   
   
       24 . The method of  claim 18 , further comprising the step of presenting responsive information to a user via the tangible user interface manipulative device. 
   
   
       25 . The method of  claim 18 , wherein the instructions relate to management of the digital content or program behavior. 
   
   
       26 . The method of  claim 18 , wherein the instructions relate to management of the behavior of the tangible user interface manipulative device.