Communication using interactive avatars

Abstract

Generally this disclosure describes a video communication system that replaces actual live images of the participating users with animated avatars. A method may include selecting an avatar; initiating communication; detecting a user input; identifying the user input; identifying an animation command based on the user input; generating avatar parameters; and transmitting at least one of the animation command and the avatar parameters.

What is claimed: 
     
         1 . A system, comprising:
 a user input device configured to capture a user input;   a communication circuitry configured to transmit and receive information; and   one or more non-transitory storage memories having stored thereon, individually or in combination, instructions that when executed by one or more processors result in the following operations comprising:   selecting an avatar;   receiving an at least one image of a user;   passively animating the avatar based at least in part on facial mapping of the at least one image, so as to produce a passively animated avatar for display on a remote device, wherein the passively animated avatar mimics motion of a body part of a user;   detecting a user input with said user input device, said user input comprising at least one of a touch and a gesture;   determining one or more animation commands associated with a user input identifier corresponding to a detected user input;   determining an interactive animation for said passively animated avatar based at least in part on said one or more animation commands;   modifying said passively animated avatar with said interactive animation, so as to produce an interactively animated avatar by deforming at least a portion of said passively animated avatar; and   transmitting a signal to said remote device, said signal configured to cause said interactively animated avatar to be displayed on said remote device.   
     
     
         2 . The system of  claim 1 , further comprising:
 a microphone configured to capture sound and convert the captured sound into a corresponding audio signal, wherein the instructions that when executed by one or more processors result in the following additional operations:   capturing user speech and converting the user speech into a corresponding user speech signal;   transforming the user speech signal into an avatar speech signal; and   transmitting the avatar speech signal.   
     
     
         3 . The system of  claim 1 , further comprising a camera configured to capture images, wherein the instructions that when executed by one or more processors result in the following additional operations:
 capturing the at least one image of the user;   performing facial detection on said at least one image to detect-a face in the image;   extracting features from the face; and   passively animating the avatar based at least in part on extracted features from said face, such that said passively animated avatar mimics motion of at least a portion of said face.   
     
     
         4 . The system of  claim 1 , further comprising a display, wherein the instructions that when executed by one or more processors result in the following additional operations:
 displaying said-avatar;   receiving at least one of a remote animation command and remote avatar parameters; and   passively animating said avatar at least in part based on at least one of the remote animation command and the remote avatar parameters.   
     
     
         5 . The system of  claim 1 , further comprising a speaker configured to convert an audio signal into sound, wherein the instructions that when executed by one or more processors result in the following additional operations:
 receiving a remote avatar speech signal; and   converting the remote avatar speech signal into avatar speech.   
     
     
         6 . The system of  claim 1 , wherein:
 the user input device is a depth camera; and   the user input is a gesture detected by said depth camera.   
     
     
         7 . The system of  claim 1 , wherein:
 the user input device is a touch-sensitive display; and   the user input is a touch event; and   said touch event comprises at least one of a touch type and a touch location.   
     
     
         8 . The system of  claim 2 , wherein the transforming comprises at least one of pitch shifting and time stretching. 
     
     
         9 . A method, comprising:
 selecting an avatar;   receiving an at least one image of a user;   passively animating the avatar based at least in part on facial mapping of the at least one image, so as to produce a passively animated avatar for display on a remote device, wherein the passively animated avatar mimics motion of a body part of a user;   detecting a user input with a user input device, said user input comprising at least one of a touch and a gesture;   determining one or more animation commands associated with a user input identifier corresponding to a detected user input;   determining an interactive animation for said passively animated avatar based at least in part on said one or more animation commands;   modifying said passively animated avatar with said interactive animation, so as to produce an interactively animated avatar by deforming at least a portion of said passively animated avatar; and   transmitting a signal to said remote device, said signal configured to cause said interactively animated avatar to be displayed on said remote device.   
     
     
         10 . The method of  claim 9 , further comprising:
 capturing user speech and converting the user speech into a corresponding user speech signal;   transforming the user speech signal into an avatar speech signal; and   transmitting the avatar speech signal.   
     
     
         11 . The method of  claim 9 , further comprising:
 capturing the at least one image of the user;   performing facial detection on said at least one image to detect-a face in the image;   extracting features from the face; and   passively animating the avatar based at least in part on extracted features from said face, such that said passively animated avatar mimics motion of at least a portion of said face.   
     
     
         12 . The method of  claim 9 , further comprising:
 displaying said-avatar;   receiving at least one of a remote animation command and remote avatar parameters; and   passively animating said avatar at least in part based on at least one of the remote animation command and the remote avatar parameters.   
     
     
         13 . The method of  claim 9 , further comprising:
 receiving a remote avatar speech signal; and   converting the remote avatar speech signal into avatar speech.   
     
     
         14 . The method of  claim 9 , wherein:
 the user input device is a depth camera; and   the user input is a gesture detected by said depth camera.   
     
     
         15 . The method of  claim 9 , wherein:
 The user input device is a touch-sensitive display;   the user input is a touch event; and   the touch event comprises at least one of a touch type and a touch location.   
     
     
         16 . The method of  claim 10 , wherein the transforming comprises at least one of pitch shifting and time stretching. 
     
     
         17 . A system comprising one or more non-transitory storage memories having stored thereon, individually or in combination, instructions that when executed by one or more processors result in the following operations comprising:
 selecting an avatar;   receiving an at least one image of a user;   passively animating the avatar based at least in part on facial mapping of the at least one image, so as to produce a passively animated avatar for display on a remote device, wherein the passively animated avatar mimics motion of a body part of a user;   detecting a user input, said user input comprising at least one of a touch and a gesture;   determining one or more animation commands associated with a user input identifier corresponding to a detected user input;   determining an interactive animation for said passively animated avatar based at least in part on said one or more animation commands;   modifying said passively animated avatar with said interactive animation, so as to produce an interactively animated avatar by deforming at least a portion of said passively animated avatar; and   transmitting a signal to said remote device, said signal configured to cause said interactively animated avatar to be displayed on said remote device.   
     
     
         18 . The system of  claim 17 , wherein the instructions that when executed by one or more processors result in the following additional operations:
 capturing user speech and converting the user speech into a corresponding user speech signal;   transforming the user speech signal into an avatar speech signal; and   transmitting the avatar speech signal.   
     
     
         19 . The system of  claim 17 , wherein the instructions that when executed by one or more processors result in the following additional operations:
 capturing the at least one image of the user;   performing facial detection on said at least one image to detect-a face in the image;   extracting features from the face; and   passively animating the avatar based at least in part on extracted features from said face, such that said passively animated avatar mimics motion of at least a portion of said face.   
     
     
         20 . The system of  claim 17 , wherein the instructions that when executed by one or more processors result in the following additional operations:
 displaying said avatar;   receiving at least one of a remote animation command and remote avatar parameters; and   passively animating said avatar at least in part based on at least one of the remote animation command and the remote avatar parameters.   
     
     
         21 . The system of  claim 17 , wherein the instructions that when executed by one or more processors result in the following additional operations:
 receiving a remote avatar speech signal; and   converting the remote avatar speech signal into avatar speech.   
     
     
         22 . The system of  claim 17 , wherein the user input is a gesture detected by a depth camera. 
     
     
         23 . The system of  claim 17 , wherein the user input is a touch event detected by a touch-sensitive display, the touch even comprising at least one of a touch type and a touch location. 
     
     
         24 . The system of  claim 17 , wherein the transforming comprises at least one of pitch shifting and time stretching.