Methods and apparatus for predicting musculo-skeletal position information using wearable autonomous sensors

Abstract

Methods and apparatus for providing a dynamically-updated computerized musculo-skeletal representation comprising a plurality of rigid body segments connected by joints. The method comprises recording, using a plurality of autonomous sensors arranged on one or more wearable devices, a plurality of autonomous signals from a user, wherein the plurality of autonomous sensors include a plurality of neuromuscular sensors configured to record neuromuscular signals. The method further comprises providing as input to a trained statistical model, the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals. The method further comprises determining, based on an output of the trained statistical model, musculo-skeletal position information describing a spatial relationship between two or more connected segments of the plurality of rigid body segments of the computerized musculo-skeletal representation, and updating the computerized musculo-skeletal representation based, at least in part, on the musculo-skeletal position information.

What is claimed is: 
     
         1 - 20 . (canceled) 
     
     
         21 . A computer-implemented method comprising:
 storing a musculo-skeletal representation of a user's body, the musculo-skeletal representation of the user's body comprising user-dependent characteristics of movements of the user's body;   measuring, using one or more sensors, signals indicative of the movements of the user's body; and   updating, in real time, the musculo-skeletal representation of the user's body to reflect the movements of the user's body by:
 providing, as input to a statistical model, information based on the signals; 
 providing, as a constraint used by the statistical model, the user-dependent characteristics of the user's movements, wherein the user-dependent characteristics of the user's movements are used by the statistical model to limit probabilities of possible spatial relationships of the user's body; 
 using the statistical model to determine, based on the user-dependent characteristics of the user's movements and the information based on the signals, a change to a spatial relationship of one or more skeletal structures in the user's body; and 
 updating the musculo-skeletal representation of the user's body to reflect the change to the spatial relationship of the one or more skeletal structures in the user's body. 
   
     
     
         22 . The computer-implemented method of  claim 21 , wherein:
 the one or more sensors comprise one or more neuromuscular sensors configured to be worn on a wrist of the user;   the signals indicative of movements of the user's body comprise neuromuscular signals measured from the user's wrist, the neuromuscular signals being indicative of movements of the user's fingers.   
     
     
         23 . The computer-implemented method of  claim 21 , wherein:
 the musculo-skeletal representation of the user's body comprises:
 information describing rigid body segments connected by joints; and 
 information describing a spatial relationship of the rigid body segments; and 
   each of the rigid body segments corresponds to one or more skeletal structures in the user's body.   
     
     
         24 . The computer-implemented method of  claim 21 , wherein the user-dependent characteristics of movements of the user's body comprise a range of motion of a joint in the user's body. 
     
     
         25 . The computer-implemented method of  claim 21 , further comprising:
 presenting a virtual-reality environment to the user, the virtual-reality environment comprising a visual representation of a character; and   updating in real time the visual representation of the character based on the updated musculo-skeletal representation of the user's body.   
     
     
         26 . The computer-implemented method of  claim 25 , wherein:
 the virtual-reality environment further comprises a virtual object; and   the computer-implemented method further comprises using the updated musculo-skeletal representation of the user's body to detect an interaction of the character with the virtual object.   
     
     
         27 . The computer-implemented method of  claim 26 , further comprising:
 measuring, using the one or more sensors, neuromuscular signals indicative of forces exerted by the user's body; and   providing, as input to the one or more statistical models, information based on the neuromuscular signals;   using the one or more statistical models to determine, based on the information based on the neuromuscular signals, a force exerted by the one or more skeletal structures in the user's body; and   updating the virtual object within the virtual-reality environment based on the force exerted by the one or more skeletal structures in the user's body.   
     
     
         28 . The computer-implemented method of  claim 26 , wherein using the updated musculo-skeletal representation of the user's body to detect the interaction of the character with the virtual object comprises detecting at least one of:
 the character grasping the virtual object;   the character dropping the virtual object;   the character pushing the virtual object;   the character throwing the virtual object;   the character pulling the virtual object;   the character opening the virtual object; or   the character closing the virtual object.   
     
     
         29 . The computer-implemented method of  claim 21 , further comprising deriving the user-dependent characteristics of movements of the user's body from measurements of an anatomy or physics of the user's body. 
     
     
         30 . The computer-implemented method of  claim 21 , further comprising deriving the user-dependent characteristics of movements of the user's body from statistical patterns of one or more observed behaviors of the user. 
     
     
         31 . A computerized system comprising:
 a memory storing a musculo-skeletal representation of a user's body, the musculo-skeletal representation of the user's body comprising user-dependent characteristics of movements of the user's body;   neuromuscular sensors configured to continuously measure neuromuscular signals indicative of the movements of the user's body; and   at least one computer processor programmed to continuously update the musculo-skeletal representation of the user's body in the memory to reflect, in real time, the movements of the user's body by:
 continuously providing, as input to one or more statistical models, information based on the neuromuscular signals; 
 continuously providing, as a constraint used by the one or more statistical models, the user-dependent characteristics of the user's movements, wherein the user-dependent characteristics of the user's movements are used by the one or more statistical models to limit probabilities of possible spatial relationships of the user's body; 
 continuously using the one or more statistical models to determine, based on the user-dependent characteristics of the user's movements and the information based on the neuromuscular signals, a change to a spatial relationship of one or more skeletal structures in the user's body; and 
 continuously updating, in the memory, the musculo-skeletal representation of the user's body to reflect the change to the spatial relationship of one or more skeletal structures in the user's body. 
   
     
     
         32 . The computerized system of  claim 31 , further comprising at least one motion sensor configured to continuously measure motion signals, wherein:
 the at least one computer processor is further programmed to provide, as additional input to the one or more statistical models, information based on the motion signals; and   the change to the spatial relationship of the one or more skeletal structures in the user's body is further determined based, at least in part, on the motion signals.   
     
     
         33 . The computerized system of  claim 32 , wherein:
 the at least one motion sensor is configured to continuously measure the motion signals at a first sampling rate;   the neuromuscular sensors are configured to continuously measure the neuromuscular signals at a second sampling rate;   the first sampling rate and the second sampling rate are different; and   at least one of the motion signals or the neuromuscular signals are resampled such that the motion signals and the neuromuscular signals are provided as input to the one or more statistical models at the same rate or the one or more statistical models are configured to process asynchronous inputs.   
     
     
         34 . The computerized system of  claim 33 , wherein the at least one motion sensor and the neuromuscular sensors are arranged on a same wearable device. 
     
     
         35 . The computerized system of  claim 31 , wherein the user-dependent characteristics of movements of the user's body are derived from behaviors of the user. 
     
     
         36 . The computerized system of  claim 31 , wherein the at least one computer processor is further programmed to send, based at least in part on the continuously updated musculo-skeletal representation, one or more control signals to a controller configured to instruct a device to perform an action based on the one or more control signals. 
     
     
         37 . The computerized system of  claim 36 , wherein:
 the controller includes a control interface; and   the control signals comprise signals to instruct at least one component of the device to move based on the change to the spatial relationship of the one or more skeletal structures in the user's body.   
     
     
         38 . The computerized system of  claim 31 , wherein the user-dependent characteristics of movements of the user's body comprise kinematic constraints. 
     
     
         39 . The computerized system of  claim 31 , further comprising a flexible or elastic band configured to be worn around an arm of the user, the neuromuscular sensors being integral to the flexible or elastic band. 
     
     
         40 . A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
 store a musculo-skeletal representation of a user's body, the musculo-skeletal representation of the user's body comprising user-dependent characteristics of movements of the user's body;   measure, using one or more neuromuscular sensors, neuromuscular signals indicative of the movements of the user's body; and   update, in real time, the musculo-skeletal representation of the user's body to reflect the movements of the user's body by:
 continuously providing, as input to one or more statistical models, information based on the neuromuscular signals; 
 continuously providing, as a constraint used by the one or more statistical models, the user-dependent characteristics of the user's movements, wherein the user-dependent characteristics of the user's movements are used by the one or more statistical models to limit probabilities of possible spatial relationships of the user's body; 
 continuously using the one or more statistical models to determine, based on the user-dependent characteristics of the user's movements and the information based on the neuromuscular signals, a change to a spatial relationship of one or more skeletal structures in the user's body; and 
 continuously updating the musculo-skeletal representation of the user's body to reflect the change to the spatial relationship of one or more skeletal structures in the user's body.