Coordinated Motion and Audio Experience Using Looped Motions

Abstract

A system for interacting with an application via a user interface is provided. A user is prompted to perform a looped motion such as an exercise move or dance move depicted by the application. User motion data is received from one or more sensors connected to a computing device executing the application. The user's progress of performing a sequence of a looped motion is determined based on the user motion data. In one embodiment, a user may perform one or more looped motions by choosing from multiple looped motions depicted by the application. In another embodiment, a user may perform looped motions at different skill levels based on a skill level determined for the user. In one embodiment, the looped motions performed by the user are displayed to the user, in an order of the looped motions performed by the user, via the user interface.

What is claimed is: 
     
         1 . A method for interacting with an application via a user interface in a computing device, comprising:
 displaying multiple looped motions via a user interface of a computing device;   receiving a choice of a looped motion from the multiple looped motions from the user, via the user interface;   receiving user motion data from a sensor connected to the computing device;   determining if the user is performing the looped motion based on the received user motion data;   if the user is performing the looped motion, determining if the user motion data matches one or more looped motion criteria;   determining the user's progress of performing a sequence of the looped motion based on the user motion data matching the one or more looped motion criteria;   determining that the user has completed the sequence of the looped motion based on the user motion data; and   displaying one or more of the sequences of looped motions performed by the user, in an order of the sequences of looped motions chosen by the user, via the user interface.   
     
     
         2 . The method of  claim 1 , further comprising:
 displaying multiple musical sequences via a user interface of a computing device, receiving a choice of a musical sequence from the multiple musical sequences from the user and synchronizing the musical sequence to the looped motion chosen by the user.   
     
     
         3 . The method of  claim 1 , wherein determining the user's progress further comprises:
 displaying the user's progress via the user interface.   
     
     
         4 . The method of  claim 3 , wherein the displaying the user's progress further comprises:
 displaying a progression tracking bar to the user and displaying a filled fraction of the progression tracking bar to the user, wherein the filled fraction indicates a percentage of completion of the sequence of the looped motion.   
     
     
         5 . The method of  claim 1 , wherein the one or more looped motion criteria comprise:
 a coordination of a musical sequence associated with the sequence of the looped motion.   
     
     
         6 . The method of  claim 1 , wherein determining if the user motion data matches one or more looped motion criteria further comprises:
 determining if a movement of a body part in the user motion data is aligned with a beat of a musical sequence associated with the sequence of the looped motion.   
     
     
         7 . The method of  claim 1 , wherein determining if the user motion data matches one or more looped motion criteria further comprises:
 determining if the energy produced through the movement of a body part in the user motion data matches the energy of a corresponding part in the sequence of the looped motion, wherein the energy is derived as a function of the mass of the body part and the speed of movement of the body part.   
     
     
         8 . The method of  claim 1 , wherein determining if the user motion data matches the one or more looped motion criteria further comprises:
 determining if a movement of a body part in the user motion data matches a pre-defined match threshold value associated with the sequence of the looped motion, wherein the pre-defined match threshold value is a measure of correlation of the user motion data with the sequence of the looped motion.   
     
     
         9 . The method of  claim 1 , wherein:
 the displaying comprises depicting an on-screen representation of a character performing the one or more sequences of looped motion and presenting the user with guided text instructing the user to perform the one or more sequences of looped motion;   the determining if the user motion data matches one or more looped motion criteria comprises determining if a movement of a body part in the user motion data is aligned with a beat of the musical sequence associated with the sequence of the looped motion, determining if the energy produced through a movement of a body part in the user motion data matches the energy of a corresponding moving point in the sequence of the looped motion and determining if a movement of a body part in the user motion data matches a match threshold value associated with sequence of the looped motion; and   the determining the user's progress of performing the sequence of the looped motion comprises displaying a progression tracking bar to the user, wherein a filled fraction of the progression tracking bar indicates a percentage of completion of the sequence of the looped motion.   
     
     
         10 . One or more processor readable storage devices having processor readable code embodied on said one or more processor readable storage devices, the processor readable code for programming one or more processors to perform a method comprising:
 displaying a looped motion via a user interface of a computing device, the looped motion coordinated to a musical sequence;   receiving user motion data from a sensor connected to the computing device;   determining if the user is performing the looped motion based on the received user motion data;   if the user is performing the looped motion, determining if the user motion data matches one or more looped motion criteria;   determining the user's progress of performing a sequence of the looped motion based on the user motion data matching the one or more looped motion criteria;;   determining that the user has completed the sequence of the looped motion based on the user motion data;   automatically determining a skill level of the user for the performed sequence of the looped motion;   choosing an additional looped motion from a set of looped motions, each looped motion of the set of looped motions is at a specific skill level, the choosing of the additional looped motions is performed based on the determined skill level of the user;   displaying the chosen additional looped motion on the user interface;   prompting the user to perform a sequence of the additional looped motion displayed on the user interface; and   displaying the one or more sequences of looped motions performed by the user, the sequences of looped motion displayed to the user in an order of the looped motions performed by the user.   
     
     
         11 . One or more processor readable storage devices according to  claim 10 , wherein:
 the one or more sequences of looped motions performed by the user are displayed to the user in an order of the skill level of the looped motions performed by the user, the skill level of the looped motions determined based on the skill level of the user.   
     
     
         12 . One or more processor readable storage devices according to  claim 10 , wherein determining the user's progress further comprises:
 displaying the user's progress via the user interface.   
     
     
         13 . One or more processor readable storage devices according to  claim 10 , wherein, the displaying the user's progress further comprises:
 displaying a progression tracking bar to the user and displaying a filled fraction of the progression tracking bar to the user, wherein the filled fraction indicates a percentage of completion of the sequence of the looped motion.   
     
     
         14 . One or more processor readable storage devices according to  claim 10 , wherein, determining the skill level of the user further comprises:
 comparing a pre-defined match threshold value associated with the sequence of the looped motion to a user match threshold value; and   determining the skill level based on the comparing.   
     
     
         15 . One or more processor readable storage devices according to  claim 14 , wherein:
 the pre-defined threshold value specifies a minimum correlation between the user motion data and the sequence of the looped motion and the user match threshold value specifies an actual correlation between the user motion data and the sequence of the looped motion.   
     
     
         16 . One or more processor readable storage devices according to  claim 15 , wherein:
 the skill level for the user is determined to be at a beginner skill level if the user match threshold value is equal to the pre-defined threshold value;   the skill level for the user is determined to be at an intermediate skill level if the user match threshold value is greater than the pre-defined threshold value by a first amount; and   the skill level for the user is determined to be at an advanced skill level if the user match threshold value is greater than the pre-defined threshold value by a second amount, the first amount and the second amount defined by a degree of increase in correlation between the user's motion and the sequence of the looped motion.   
     
     
         17 . An apparatus to perform interaction with an application via a user interface in a computing device, comprising:
 a depth camera to capture user motion data from a capture device;   a computing device connected to the depth camera to receive the user motion data, wherein the computing device displays a looped motion via a user interface to a user, determines if the user is performing the looped motion based on the received user motion data, determines if the user motion data matches one or more looped motion criteria and dynamically alters the looped motion when the user motion data does not match the one or more looped motion criteria.   
     
     
         18 . The apparatus of  claim 17 , wherein:
 the computing device determines the user's progress of performing the looped motion based on the user motion data matching the one or more looped motion criteria; and   the computing device determines that the user has completed the sequence of the looped motion based on the user's progress of performing the sequence of the looped motion.   
     
     
         19 . The apparatus of  claim 18 , wherein the one or more looped motion criteria comprise:
 a coordination of a musical sequence associated with the sequence of the looped motion.   
     
     
         20 . The apparatus of  claim 18  further comprising:
 a display device connected to the computing device, the display device displays the user's progress via a progression tracking bar to the user, wherein a filled fraction indicates a percentage of completion of the sequence of the looped motion.