Automatic avatar creation

Abstract

A three-dimensional (“3D”) avatar can be automatically created that resembles the physical appearance of an individual captured in one or more input images or video frames. The avatar can be further customized by the individual in an editing environment and used in various applications, including but not limited to gaming, social networking and video conferencing.

What is claimed is: 
     
       1. A computer implemented method, comprising:
 receiving one or more digital images of an individual;   determining, in a computer, digital representations of facial elements of the individual from the one or more digital images;   searching a population of avatars using a genetic process, where the genetic process includes using the computer to evaluate the overall fitness of the avatars using a fitness function that compares the digital representations of the individual's facial elements to digital representations of corresponding avatar facial elements; and   selecting or receiving a selection of one or more avatars from the population of avatars based on fitness values associated with the avatars;   wherein evaluating the overall fitness of the avatars using a fitness function includes calculating fitness values for individual facial elements of the avatars based on linear and geometric weights corresponding to each of the individual facial elements.   
     
     
       2. The method of  claim 1 , where determining digital representations of facial elements of the individual, further comprises:
 image processing the input images to identify the individual's facial elements; and   generating binary masks or images for the identified facial elements.   
     
     
       3. The method of  claim 1 , where the digital representations of the corresponding avatar facial elements are binary masks. 
     
     
       4. The method of  claim 1 , further comprising:
 initializing the genetic process by randomly generating a population of avatars having the facial elements.   
     
     
       5. The method of  claim 1 , further comprising:
 initializing the genetic process by seeding the avatar population with avatars having the facial elements.   
     
     
       6. The method of  claim 1 , where evaluating the overall fitness of the avatars using a fitness function, further comprises: calculating the overall fitness values for the avatars based on the individual fitness values. 
     
     
       7. The method of  claim 6 , where the overall fitness function is given by 
       
         
           
             
               
                 f 
                 = 
                 
                   
                     ∑ 
                     i 
                   
                   
                     
                       α 
                       i 
                     
                     · 
                     
                       
                         f 
                           
                       
                       i 
                       
                         γ 
                         i 
                       
                     
                   
                 
               
               , 
             
           
         
         where α i  and γ i  are linear and geometric weights for each facial element fitness value f i   γ     i   , respectively, and i is an index into the facial elements. 
       
     
     
       8. The method of  claim 1 , where the facial elements are from a group of facial elements including face, eyebrows, eyes, nose, mouth, chin and hair. 
     
     
       9. The method of  claim 1 , further comprising:
 preprocessing the input images using one or more of the following processes: resizing, color space conversion, inverted square root y channel, fine texture, cartoon texture and face detection.   
     
     
       10. The method of  claim 1 , further comprising:
 presenting the selected one or more avatars on a touch sensitive display; and   receiving input from the touch sensitive display selecting an avatar.   
     
     
       11. A system, comprising:
 a processor; and   a computer-readable storage medium coupled to the processor and including instructions, which, when executed by the processor, causes the processor to perform operations comprising:   receiving one or more digital images of an individual;   determining, in a computer, digital representations of facial elements of the individual from the one or more digital images;   searching a population of avatars using a genetic process, where the genetic process includes using the computer to evaluate the overall fitness of the avatars using a fitness function that compares the digital representations of the individual's facial elements to digital representations of corresponding avatar facial elements; and   selecting or receiving a selection of one or more avatars from the population of avatars based on fitness values associated with the avatars;   wherein evaluating the overall fitness of the avatars using a fitness function includes calculating fitness values for individual facial elements of the avatars based on linear and geometric weights corresponding to each of the individual facial elements.   
     
     
       12. The system of  claim 11 , where determining digital representations of facial elements of the individual, further comprises:
 image processing the input images to identify the individual's facial elements; and   generating binary masks or images for the identified facial elements.   
     
     
       13. The system of  claim 11 , where the digital representations of the corresponding avatar facial elements are binary masks. 
     
     
       14. The system of  claim 11 , where evaluating the overall fitness of the avatars using a fitness function, further comprises: calculating the overall fitness values for the avatars based on the individual fitness values. 
     
     
       15. The system of  claim 11 , further comprising:
 preprocessing the input images using one or more of the following processes: resizing, color space conversion, inverted square root y channel, fine texture, cartoon texture and face detection.   
     
     
       16. The system of  claim 11 , further comprising:
 presenting the selected one or more avatars on a touch sensitive display; and   receiving input from the touch sensitive display selecting an avatar.   
     
     
       17. A non-transitory computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, causes the one or more processors to perform operations comprising:
 receiving one or more digital images of an individual;   determining, in a computer, digital representations of facial elements of the individual from the one or more digital images;   searching a population of avatars using a genetic process, where the genetic process includes using the computer to evaluate the overall fitness of the avatars using a fitness function that compares the digital representations of the individual's facial elements to digital representations of corresponding avatar facial elements; and   selecting or receiving a selection of one or more avatars from the population of avatars based on fitness values associated with the avatars;   wherein evaluating the overall fitness of the avatars using a fitness function includes calculating fitness values for individual facial elements of the avatars based on linear and geometric weights corresponding to each of the individual facial elements.   
     
     
       18. The non-transitory computer-readable storage medium of  claim 17 , where determining digital representations of facial elements of the individual, further comprises:
 image processing the input images to identify the individual's facial elements; and   generating binary masks or images for the identified facial elements.   
     
     
       19. The non-transitory computer-readable storage medium of  claim 17 , where the digital representations of the corresponding avatar facial elements are binary masks. 
     
     
       20. The non-transitory computer-readable storage medium of  claim 17 , where evaluating the overall fitness of the avatars using a fitness function, further comprises: calculating the overall fitness values for the avatars based on the individual fitness values. 
     
     
       21. A method, comprising: 
       at an electronic device in communication with a display and one or more cameras:
 concurrently displaying, via the display, a representation of video captured by the one or more cameras, one or more selectable options selectable to edit avatar features, and a preview of a virtual avatar overlaid over the representation of the video, wherein the preview of the virtual avatar includes a representation of a head of a user of the electronic device, and one or more features of the representation of the head include facial elements that are generated based on the video captured by the one or more cameras of the electronic device; 
 while concurrently displaying the one or more selectable options and the preview of the virtual avatar overlaid over the representation of the video, detecting movement of the head of the user; and 
 in response to detecting the movement of the head of the user, varying a posture of the representation of the head of the virtual avatar in accordance with the movement of the head of the user.  
 
     
     
       22. The method of  claim 21 , wherein the representation of the head of the user is displayed with a first level of transparency.  
     
     
       23. The method of  claim 21 , further comprising:
 while concurrently displaying the one or more selectable options and the preview of the virtual avatar overlaid over the representation of the video, receiving a user input; and   in response to the user input, modifying a first visual characteristic of the virtual avatar based on the user input.    
     
     
       24. The method of  claim 21 , wherein the representation of video captured by the one or more cameras, the one or more selectable options selectable to edit the avatar features, and the preview of the virtual avatar overlaid over the representation of the video are concurrently displayed during an avatar creation process.  
     
     
       25. The method of  claim 24 , wherein the representation of video corresponds to currently captured video via the one or more cameras.  
     
     
       26. The method of  claim 25 , wherein the one or more selectable options are associated with the one or more features of the representation of the head, the method further comprising:
 while displaying the one or more selectable options, receiving, via one or more input devices, an input selecting a respective selectable option of the one or more selectable options; and   in response to receiving the input selecting the respective selectable option, displaying a customization user interface for modifying the representation of the head.    
     
     
       27. A non-transitory computer-readable medium storing instructions, which when executed by one or more processors of an electronic device in communication with a display, a camera, and one or more input devices, cause the one or more processors to perform a method comprising:
 concurrently displaying, via the display, a representation of video captured by the one or more cameras, one or more selectable options selectable to edit avatar features, and a preview of a virtual avatar overlaid over the representation of the video, wherein the preview of the virtual avatar includes a representation of a head of a user of the electronic device, and one or more features of the representation of the head include facial elements that are generated based on the video captured by the one or more cameras of the electronic device;   while concurrently displaying the one or more selectable options and the preview of the virtual avatar overlaid over the representation of the video, detecting movement of the head of the user; and   in response to detecting the movement of the head of the user, varying a posture of the representation of the head of the virtual avatar in accordance with the movement of the head of the user.    
     
     
       28. An electronic device in communication with a display, a camera, and one or more input devices comprising:
 one or processors; and   memory storing instructions, which when executed by the one or more processors, cause the one or more processors to perform a method comprising:   concurrently displaying, via the display, a representation of video captured by the one or more cameras, one or more selectable options selectable to edit avatar features, and a preview of a virtual avatar overlaid over the representation of the video, wherein the preview of the virtual avatar includes a representation of a head of a user of the electronic device, and one or more features of the representation of the head include facial elements that are generated based on the video captured by the one or more cameras of the electronic device;   while concurrently displaying the one or more selectable options and the preview of the virtual avatar overlaid over the representation of the video, detecting movement of the head of the user; and   in response to detecting the movement of the head of the user, varying a posture of the representation of the head of the virtual avatar in accordance with the movement of the head of the user.    
     
     
       29. The non-transitory computer-readable medium of  claim 27 , wherein the representation of the head of the user is displayed with a first level of transparency. 
     
     
       30. The non-transitory computer-readable medium of  claim 27 , the method further comprising:
 while concurrently displaying the one or more selectable options and the preview of the virtual avatar overlaid over the representation of the video, receiving a user input; and   in response to the user input, modifying a first visual characteristic of the virtual avatar based on the user input.   
     
     
       31. The non-transitory computer-readable medium of  claim 27 , wherein the representation of video captured by the one or more cameras, the one or more selectable options selectable to edit the avatar features, and the preview of the virtual avatar overlaid over the representation of the video are concurrently displayed during an avatar creation process. 
     
     
       32. The non-transitory computer-readable medium of  claim 31 , wherein the representation of video corresponds to currently captured video via the one or more cameras. 
     
     
       33. The non-transitory computer-readable medium of  claim 32 , wherein the one or more selectable options are associated with the one or more features of the representation of the head, the method further comprising:
 while displaying the one or more selectable options, receiving, via one or more input devices, an input selecting a respective selectable option of the one or more selectable options; and   in response to receiving the input selecting the respective selectable option, displaying a customization user interface for modifying the representation of the head.   
     
     
       34. The electronic device of  claim 28 , wherein the representation of the head of the user is displayed with a first level of transparency. 
     
     
       35. The electronic device of  claim 28 , the method further comprising:
 while concurrently displaying the one or more selectable options and the preview of the virtual avatar overlaid over the representation of the video, receiving a user input; and   in response to the user input, modifying a first visual characteristic of the virtual avatar based on the user input.   
     
     
       36. The electronic device of  claim 28 , wherein the representation of video captured by the one or more cameras, the one or more selectable options selectable to edit the avatar features, and the preview of the virtual avatar overlaid over the representation of the video are concurrently displayed during an avatar creation process. 
     
     
       37. The electronic device of  claim 36 , wherein the representation of video corresponds to currently captured video via the one or more cameras. 
     
     
       38. The electronic device of  claim 37 , wherein the one or more selectable options are associated with the one or more features of the representation of the head, the method further comprising:
 while displaying the one or more selectable options, receiving, via one or more input devices, an input selecting a respective selectable option of the one or more selectable options; and   in response to receiving the input selecting the respective selectable option, displaying a customization user interface for modifying the representation of the head.