System and method for communication using interactive avatar

Abstract

A video communication system that replaces actual live images of the participating users with animated avatars. A method may include selecting an avatar, initiating communication, capturing an image, detecting a face in the image, determining facial characteristics from the face, including eye movement and eyelid movement of a user indicative of direction of user gaze and blinking, respectively, converting the facial features to avatar parameters, and transmitting at least one of the avatar selection or avatar parameters.

What is claimed is: 
     
         1 . At least one device to communicate using avatars, comprising:
 avatar selection circuitry to receive from a user a selection of an avatar from a plurality of predefined avatars, the selected avatar to be animated on a remote device;   communication circuitry to transmit the avatar selection to the remote device;   camera and audio framework circuitry to capture at least one image; and   face detection circuitry to track the user's face in the at least one image and identify one or more facial characteristics in the face, the face detection circuitry including at least:
 eye detection and tracking circuitry to determine current eye status based on the at least one image, determine changes in eye status based on the current eye status and formulate at least one instruction to cause the avatar to mimic any changes in eye status that were determined, the at least one instruction being transmitted to the remote device via the communication circuitry for use in animating of the avatar. 
   
     
     
         2 . The at least one device of  claim 1 , wherein the camera and audio framework circuitry is coupled to at least one camera to capture the at least one image, at least one microphone to capture local sounds and at least one speaker to reproduce the local sounds to provide audio feedback to the user and to reproduce remote sounds captured by the remote device. 
     
     
         3 . The at least one device of  claim 1 , wherein the face detection circuitry is further to recognize an expression by classifying the one or more facial characteristics, the at least one instruction at least indicating the expression to the remote device for use in animating the avatar. 
     
     
         4 . The at least one device of  claim 3 , wherein the at least one instruction is to cause the avatar to display an exaggerated version of the expression to emphasize the expression. 
     
     
         5 . The at least one device of  claim 1 , further comprising a display to display at least a feedback avatar replicating the animation of the avatar on the remote device. 
     
     
         6 . The at least one device of  claim 5 , wherein the at least one instruction is to cause the avatar to be animated on the remote device in a virtual space and the display is further to display a perspective view of what the avatar sees in the virtual space. 
     
     
         7 . The at least one device of  claim 1 , wherein the at least one instruction is specific to the selected avatar. 
     
     
         8 . The at least one device of  claim 7 , wherein upon selection of a new avatar the avatar selection circuitry is to cause the communication circuitry to transmit the new avatar selection to the remote device and alter the at least one instruction based on the new avatar. 
     
     
         9 . The at least one device of  claim 1 , wherein the face detection circuitry further comprises face normalization circuitry to normalize a face detected in the at least one image. 
     
     
         10 . The at least one device of  claim 9 , wherein normalization comprises at least rotating the at least one image to align eyes within a face captured in the image, cropping the at least one image corresponding to a size of the face, scaling the at least one image to make a distance between the eyes constant, applying a mask to zero out pixels not within an oval that would contain a typical face, histogram equalize the at least one image to smooth the distribution of gray values for non-masked pixels, and normalizing the at least one image so the non-masked pixels have zero mean and a standard deviation of one. 
     
     
         11 . The at least one device of  claim 9 , wherein the eye detection and tracking circuitry comprises eye classification circuitry to determine the current eye status including at least one of eye opening, eye closing or eye movement based on at least one normalized image received from the face normalization circuitry. 
     
     
         12 . A method for communicating using avatars, comprising:
 receiving from a user, via avatar selection circuitry, a selection of an avatar from a plurality of predefined avatars, the selected avatar to be animated on a remote device;   transmitting, using communication circuitry, the avatar selection to the remote device;   capturing, using camera and audio framework circuitry, at least one image;   tracking, using face detection circuitry, the user's face in the at least one image and identifying one or more facial characteristics in the face;   determining, using eye detection and tracking circuitry in the face detection circuitry, current eye status based on the at least one image, determining changes in eye status based on the current eye status and formulating at least one instruction to cause the avatar to mimic any changes in eye status that were determined, the at least one instruction being transmitted to the remote device via the communication circuitry for use in animating of the avatar.   
     
     
         13 . The method of  claim 12 , further comprising:
 capturing, using at least one camera coupled to the camera and audio framework circuitry, the at least one image;   capturing, using at least one microphone coupled to the camera and audio framework circuitry, local sounds;   reproducing, using at least one speaker coupled to the camera and audio framework circuitry, the local sounds to provide audio feedback to the user and remote sounds captured by the remote device.   
     
     
         14 . The method of  claim 12 , further comprising:
 recognizing, using the face detection circuitry, an expression by classifying the one or more facial characteristics, the at least one instruction at least indicating the expression to the remote device for use in animating the avatar.   
     
     
         15 . The method of  claim 12 , further comprising:
 displaying, using a display, at least a feedback avatar replicating the animation of the avatar on the remote device.   
     
     
         16 . The method of  claim 12 , further comprising:
 normalizing, using face normalization circuitry in the face detection circuitry, a face detected in the at least one image.   
     
     
         17 . The method of  claim 16 , wherein normalizing comprises at least rotating the at least one image to align eyes within a face captured in the image, cropping the at least one image corresponding to a size of the face, scaling the at least one image to make a distance between the eyes constant, applying a mask to zero out pixels not within an oval that would contain a typical face, histogram equalize the at least one image to smooth the distribution of gray values for non-masked pixels, and normalizing the at least one image so the non-masked pixels have zero mean and a standard deviation of one. 
     
     
         18 . The method  claim 16 , further comprising:
 determining, using eye classification circuitry in the eye detection and tracking circuitry, the current eye status including at least one of eye opening, eye closing or eye movement based on at least one normalized image received from the face normalization circuitry.   
     
     
         19 . At least one machine-readable storage medium having stored thereon, individually or in combination, instructions for communicating using avatars that, when executed by one or more processors, cause the one or more processors to:
 receive from a user, via avatar selection circuitry, a selection of an avatar from a plurality of predefined avatars, the selected avatar to be animated on a remote device;   transmit, using communication circuitry, the avatar selection to the remote device;   capture, using camera and audio framework circuitry, at least one image;   track, using face detection circuitry, the user's face in the at least one image and identifying one or more facial characteristics in the face;   determine, using eye detection and tracking circuitry in the face detection circuitry, current eye status based on the at least one image, determine changes in eye status based on the current eye status and formulate at least one instruction to cause the avatar to mimic any changes in eye status that were determined, the at least one instruction being transmitted to the remote device via the communication circuitry for use in animating of the avatar.   
     
     
         20 . The storage medium of  claim 19 , further comprising instructions for communicating using avatars that, when executed by one or more processors, cause the one or more processors to:
 capture, using at least one camera coupled to the camera and audio framework circuitry, the at least one image;   capture, using at least one microphone coupled to the camera and audio framework circuitry, local sounds;   reproduce, using at least one speaker coupled to the camera and audio framework circuitry, the local sounds to provide audio feedback to the user and remote sounds captured by the remote device.   
     
     
         21 . The storage medium of  claim 19 , further comprising instructions for communicating using avatars that, when executed by one or more processors, cause the one or more processors to:
 recognize, using the face detection circuitry, an expression by classifying the one or more facial characteristics, the at least one instruction at least indicating the expression to the remote device for use in animating the avatar.   
     
     
         22 . The storage medium of  claim 19 , further comprising instructions for communicating using avatars that, when executed by one or more processors, cause the one or more processors to:
 display, using a display, at least a feedback avatar replicating the animation of the avatar on the remote device.   
     
     
         23 . The storage medium of  claim 19 , further comprising instructions for communicating using avatars that, when executed by one or more processors, cause the one or more processors to:
 normalize, using face normalization circuitry in the face detection circuitry, a face detected in the at least one image.   
     
     
         24 . The storage medium of  claim 23 , wherein the instructions to normalize comprise instructions to at least rotate the at least one image to align eyes within a face captured in the image, crop the at least one image corresponding to a size of the face, scale the at least one image to make a distance between the eyes constant, apply a mask to zero out pixels not within an oval that would contain a typical face, histogram equalize the at least one image to smooth the distribution of gray values for non-masked pixels, and normalize the at least one image so the non-masked pixels have zero mean and a standard deviation of one. 
     
     
         25 . The storage medium of  claim 23 , further comprising instructions for communicating using avatars that, when executed by one or more processors, cause the one or more processors to:
 determine, using eye classification circuitry in the eye detection and tracking circuitry, the current eye status including at least one of eye opening, eye closing or eye movement based on at least one normalized image received from the face normalization circuitry.