Speech-controlled animation system

Abstract

Methods, systems and apparatuses directed toward an authoring tool that gives users the ability to make high-quality, speech-driven animation in which the animated character speaks in the user's voice. Embodiments of the present invention allow the animation to be sent as a message over the Internet or used as a set of instructions for various applications including Internet chat rooms. According to one embodiment, the user chooses a character and a scene from a menu, then speaks into the computer's microphone to generate a personalized message. Embodiments of the present invention use voice-recognition technology to match the audio input to the appropriate animated mouth shapes creating a professional looking 2D or 3D animated scene with lip-synced audio characteristics.

1 .- 33 . (cancelled)  
     
     
         34 . A method for generating an animated sequence having synchronized visual and audio characteristics during the play back of audio data, said method executed in a computing device including a mouth shape database including a plurality of mouth shapes corresponding to events and an image frame database storing a plurality of image frames, at least one of said image frames including an animated character, said method comprising the steps of 
 (a) receiving audio data;    (b) detecting a phonetic code sequence in said audio data;    (c) generating an event sequence from said phonetic code sequence; and    during the play back of said audio data:    (d) tracking the audio playback time in said step (d);    (e) sampling said event sequence using said playback time tracked in step (d);    (f) constructing an animation frame based on an image frame selected from the image frame database and the mouth shape corresponding to the event sampled in said sampling step (e);    (g) displaying the animation frame; and    (h) repeating steps (e)-(g) a desired number of times.    
     
     
         35 . The method of  claim 34  wherein steps (e)-(g) are repeated for the duration of said audio data.  
     
     
         36 . The method of  claim 34  wherein said event sequence sampled in step (f) is sampled at a predetermined interval from said playback time detected in said step (e).  
     
     
         37 . The method of  claim 34  further comprising 
 (i) monitoring the delay associated with the constructing and displaying of animation frames in steps (f) and (g);  
 and wherein the sampling step (e) is based on said playback time tracked in step (d) and said delay monitored in step (i).  
 
     
     
         38 .- 45 . (cancelled)  
     
     
         46 . A method for driving a user interface displaying at least one animated character, said method comprising the steps of 
 (a) receiving a at least one packet, said at least one packet comprising audio data and a phonetic code sequence;    (b) generating an event sequence using said phonetic code sequence;    (c) playing back said audio data;    (d) tracking the audio playback time;    (e) sampling said event sequence using said playback time tracked in step (d);    (f) displaying an animation frame based on said sampling step (e); and    (g) repeating steps (d)-(g) a desired number of times.    
     
     
         47 . The method of  claim 46  wherein steps (d)-(g) are repeated for the duration of said audio data.  
     
     
         48 . The method of  claim 46  wherein said event sequence sampled in step (e) is sampled at a predetermined interval from said playback time detected in said step (d).  
     
     
         49 . The method of  claim 46  further comprising 
 (h) monitoring the delay associated with the constructing and displaying of animation frames in steps (f) and (g);  
 and wherein the sampling step (e) is based on said playback time tracked in step (d) and said delay monitored in step (h).  
 
     
     
         50 . A method for driving a user interface displaying at least one animated character, said method comprising the steps of 
 (a) receiving a at least one packet, said at least one packet comprising audio data and an event sequence;    (b) playing back said audio data;    (c) tracking the audio playback time in said step (d);    (d) sampling said event sequence using said playback time tracked in step (c);    (e) displaying an animation frame based on said sampling step (d); and    (f) repeating steps (c)-(f) a desired number of times.    
     
     
         51 . The method of  claim 50  wherein steps (c)-(f) are repeated for the duration of said audio data.  
     
     
         52 . The method of  claim 50  wherein said event sequence sampled in step (d) is sampled at a predetermined interval from said playback time detected in said step (c).  
     
     
         53 . The method of  claim 50  further comprising 
 (h) monitoring the delay associated with the constructing and displaying of animation frames in steps (e) and (f);  
 and wherein the sampling step (d) is based on said playback time tracked in step (c) and said delay monitored in step (h).