Automatic avatar creation

Abstract

A three-dimensional (“3D”) avatar can be automatically created that resembles the physical appearance of an individual captured in one or more input images or video frames. The avatar can be further customized by the individual in an editing environment and used in various applications, including but not limited to gaming, social networking and video conferencing.

What is claimed is: 
     
       1. A computer implemented method, comprising:
 receiving one or more digital images of an individual;   determining, in a computer, digital representations of facial elements of the individual from the one or more digital images;   searching a population of avatars using a genetic process, where the genetic process includes using the computer to evaluate the overall fitness of the avatars using a fitness function that compares the digital representations of the individual's facial elements to digital representations of corresponding avatar facial elements; and   selecting or receiving a selection of one or more avatars from the population of avatars based on fitness values associated with the avatars;   wherein evaluating the overall fitness of the avatars using a fitness function includes calculating fitness values for individual facial elements of the avatars based on linear and geometric weights corresponding to each of the individual facial elements.   
     
     
       2. The method of  claim 1 , where determining digital representations of facial elements of the individual, further comprises:
 image processing the input images to identify the individual's facial elements; and   generating binary masks or images for the identified facial elements.   
     
     
       3. The method of  claim 1 , where the digital representations of the corresponding avatar facial elements are binary masks. 
     
     
       4. The method of  claim 1 , further comprising:
 initializing the genetic process by randomly generating a population of avatars having the facial elements.   
     
     
       5. The method of  claim 1 , further comprising:
 initializing the genetic process by seeding the avatar population with avatars having the facial elements.   
     
     
       6. The method of  claim 1 , where evaluating the overall fitness of the avatars using a fitness function, further comprises: calculating the overall fitness values for the avatars based on the individual fitness values. 
     
     
       7. The method of  claim 6 , where the overall fitness function is given by 
       
         
           
             
               
                 f 
                 = 
                 
                   
                     ∑ 
                     i 
                   
                   ⁢ 
                   
                     
                       α 
                       i 
                     
                     · 
                     
                       f 
                       i 
                       
                         γ 
                         i 
                       
                     
                   
                 
               
               , 
             
           
         
         where α i  and γ i  are linear and geometric weights for each facial element fitness value f i   γ     i   , respectively, and i is an index into the facial elements. 
       
     
     
       8. The method of  claim 1 , where the facial elements are from a group of facial elements including face, eyebrows, eyes, nose, mouth, chin and hair. 
     
     
       9. The method of  claim 1 , further comprising:
 preprocessing the input images using one or more of the following processes: resizing, color space conversion, inverted square root y channel, fine texture, cartoon texture and face detection.   
     
     
       10. The method of  claim 1 , further comprising:
 presenting the selected one or more avatars on a touch sensitive display; and   receiving input from the touch sensitive display selecting an avatar.   
     
     
       11. A system, comprising:
 a processor; and   a computer-readable storage medium coupled to the processor and including instructions, which, when executed by the processor, causes the processor to perform operations comprising:   receiving one or more digital images of an individual;   determining, in a computer, digital representations of facial elements of the individual from the one or more digital images;   searching a population of avatars using a genetic process, where the genetic process includes using the computer to evaluate the overall fitness of the avatars using a fitness function that compares the digital representations of the individual's facial elements to digital representations of corresponding avatar facial elements; and   selecting or receiving a selection of one or more avatars from the population of avatars based on fitness values associated with the avatars;   wherein evaluating the overall fitness of the avatars using a fitness function includes calculating fitness values for individual facial elements of the avatars based on linear and geometric weights corresponding to each of the individual facial elements.   
     
     
       12. The system of  claim 11 , where determining digital representations of facial elements of the individual, further comprises:
 image processing the input images to identify the individual's facial elements; and   generating binary masks or images for the identified facial elements.   
     
     
       13. The system of  claim 11 , where the digital representations of the corresponding avatar facial elements are binary masks. 
     
     
       14. The system of  claim 11 , where evaluating the overall fitness of the avatars using a fitness function, further comprises: calculating the overall fitness values for the avatars based on the individual fitness values. 
     
     
       15. The system of  claim 11 , further comprising:
 preprocessing the input images using one or more of the following processes: resizing, color space conversion, inverted square root y channel, fine texture, cartoon texture and face detection.   
     
     
       16. The system of  claim 11 , further comprising:
 presenting the selected one or more avatars on a touch sensitive display; and   receiving input from the touch sensitive display selecting an avatar.   
     
     
       17. A non-transitory computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, causes the one or more processors to perform operations comprising:
 receiving one or more digital images of an individual;   determining, in a computer, digital representations of facial elements of the individual from the one or more digital images;   searching a population of avatars using a genetic process, where the genetic process includes using the computer to evaluate the overall fitness of the avatars using a fitness function that compares the digital representations of the individual's facial elements to digital representations of corresponding avatar facial elements; and   selecting or receiving a selection of one or more avatars from the population of avatars based on fitness values associated with the avatars;   wherein evaluating the overall fitness of the avatars using a fitness function includes calculating fitness values for individual facial elements of the avatars based on linear and geometric weights corresponding to each of the individual facial elements.   
     
     
       18. The non-transitory computer-readable storage medium of  claim 17 , where determining digital representations of facial elements of the individual, further comprises:
 image processing the input images to identify the individual's facial elements; and   generating binary masks or images for the identified facial elements.   
     
     
       19. The non-transitory computer-readable storage medium of  claim 17 , where the digital representations of the corresponding avatar facial elements are binary masks. 
     
     
       20. The non-transitory computer-readable storage medium of  claim 17 , where evaluating the overall fitness of the avatars using a fitness function, further comprises: calculating the overall fitness values for the avatars based on the individual fitness values. 
     
     
       21. A method, comprising:
 at an electronic device with a display, a camera, and one or more input devices:
 displaying, on the display, a first user interface that provides information about generation, by the electronic device, of an avatar of a user of the electronic device; 
 while displaying the first user interface, receiving, via the one or more input devices, a request to initiate the generation of the avatar based on a digital representation of the user of the electronic device that is being captured by the camera of the electronic device; 
 in response to receiving the request to initiate the generation of the avatar, while capturing the digital representation of the user, displaying, on the display, a second user interface that includes:
 the digital representation of the user as it is being captured by the camera of the electronic device, and 
 a visual indication of a current status of evaluating the digital representation of the user of the electronic device that is being captured by the camera of the electronic device, wherein:
 the visual indication is overlaid on the digital representation of the user as the digital representation of the user is being evaluated, and 
 the visual indication moves from being overlaid on a first portion of the digital representation of the user corresponding to a first facial element to being overlaid on a second portion, different from the first portion, of the digital representation of the user corresponding to a second facial element, different from the first facial element, as the digital representation of the user is being evaluated; 
 
 
 after displaying the second user interface, displaying, on the display, a first proposed avatar based on the digital representation of the user; and 
 while displaying the first proposed avatar, providing the user with options for customizing the first proposed avatar.  
   
     
     
       22. The method of claim 21, further comprising:
 after displaying the second user interface, concurrently displaying, on the display, the first proposed avatar and a second proposed avatar based on the digital representation of the user.    
     
     
       23. The method of claim 22, further comprising:
 while concurrently displaying the first proposed avatar and the second proposed avatar on the display, receiving, via the one or more input devices, an input corresponding to a selection of a respective proposed avatar of the first and second proposed avatars; and   in response to receiving the input, entering a manual editing mode on the electronic device for manually editing the respective proposed avatar.    
     
     
       24. The method of claim 21, wherein displaying the second user interface includes displaying dynamic feedback for guiding a position of the user that will improve an accuracy of the avatar that is being generated by the electronic device, and the feedback for guiding the position of the user comprises feedback for constraining the position of the user.  
     
     
       25. The method of claim 24, wherein the feedback for constraining the position of the user comprises feedback for constraining a position of the user's head while the camera captures successive images of the user.  
     
     
       26. The method of claim 21, further comprising:
 while displaying the first proposed avatar on the display, receiving, via the one or more input devices, a touch input from the user with respect to the first proposed avatar; and   in response to receiving the touch input, performing a corresponding action on the electronic device with respect to the first proposed avatar.    
     
     
       27. The method of claim 26, wherein the touch input with respect to the first proposed avatar comprises touching of the first proposed avatar on the display.  
     
     
       28. The method of claim 26, wherein the corresponding action comprises changing a display characteristic of the first proposed avatar on the display.  
     
     
       29. The method of claim 28, wherein changing the display characteristic of the first proposed avatar on the display comprises enlarging the first proposed avatar on the display.  
     
     
       30. The method of claim 21, wherein the visual indication of the current status indicates a respective portion of the user currently being evaluated.  
     
     
       31. The method of claim 21, wherein the second user interface includes a latest candidate avatar of the user, overlaid on the digital representation of the user as it is being captured by the camera, based on a current progress of the evaluation of the digital representation of the user.  
     
     
       32. A non-transitory computer-readable medium storing instructions, which when executed by one or more processors of an electronic device with a display, a camera, and one or more input devices, cause the one or more processors to perform a method comprising:
 displaying, on the display, a first user interface that provides information about generation, by the electronic device, of an avatar of a user of the electronic device;   while displaying the first user interface, receiving, via the one or more input devices, a request to initiate the generation of the avatar based on a digital representation of the user of the electronic device that is being captured by the camera of the electronic device;   in response to receiving the request to initiate the generation of the avatar, while capturing the digital representation of the user, displaying, on the display, a second user interface that includes:
 the digital representation of the user as it is being captured by the camera of the electronic device, and 
 a visual indication of a current status of evaluating the digital representation of the user of the electronic device that is being captured by the camera of the electronic device, wherein:
 the visual indication is overlaid on the digital representation of the user as the digital representation of the user is being evaluated, and 
 the visual indication moves from being overlaid on a first portion of the digital representation of the user corresponding to a first facial element to being overlaid on a second portion, different from the first portion, of the digital representation of the user corresponding to a second facial element, different from the first facial element, as the digital representation of the user is being evaluated; 
 
   after displaying the second user interface, displaying, on the display, a first proposed avatar based on the digital representation of the user; and   while displaying the first proposed avatar, providing the user with options for customizing the first proposed avatar.    
     
     
       33. The non-transitory computer-readable medium of claim 32, the method further comprising:
 after displaying the second user interface, concurrently displaying, on the display, the first proposed avatar and a second proposed avatar based on the digital representation of the user.   
     
     
       34. The non-transitory computer-readable medium of claim 33, the method further comprising:
 while concurrently displaying the first proposed avatar and the second proposed avatar on the display, receiving, via the one or more input devices, an input corresponding to a selection of a respective proposed avatar of the first and second proposed avatars; and   in response to receiving the input, entering a manual editing mode on the electronic device for manually editing the respective proposed avatar.   
     
     
       35. The non-transitory computer-readable medium of claim 32, wherein displaying the second user interface includes displaying dynamic feedback for guiding a position of the user that will improve an accuracy of the avatar that is being generated by the electronic device, and the feedback for guiding the position of the user comprises feedback for constraining the position of the user. 
     
     
       36. The non-transitory computer-readable medium of claim 35, wherein the feedback for constraining the position of the user comprises feedback for constraining a position of the user's head while the camera captures successive images of the user. 
     
     
       37. The non-transitory computer-readable medium of claim 32, the method further comprising:
 while displaying the first proposed avatar on the display, receiving, via the one or more input devices, a touch input from the user with respect to the first proposed avatar; and   in response to receiving the touch input, performing a corresponding action on the electronic device with respect to the first proposed avatar.   
     
     
       38. The non-transitory computer-readable medium of claim 37, wherein the touch input with respect to the first proposed avatar comprises touching of the first proposed avatar on the display. 
     
     
       39. The non-transitory computer-readable medium of claim 37, wherein the corresponding action comprises changing a display characteristic of the first proposed avatar on the display. 
     
     
       40. The non-transitory computer-readable medium of claim 39, wherein changing the display characteristic of the first proposed avatar on the display comprises enlarging the first proposed avatar on the display. 
     
     
       41. The non-transitory computer-readable medium of claim 32, wherein the visual indication of the current status indicates a respective portion of the user currently being evaluated. 
     
     
       42. The non-transitory computer-readable medium of claim 32, wherein the second user interface includes a latest candidate avatar of the user, overlaid on the digital representation of the user as it is being captured by the camera, based on a current progress of the evaluation of the digital representation of the user. 
     
     
       43. An electronic device comprising:
 one or more processors;   a display;   a camera;   one or more input devices; and   memory storing instructions, which when executed by the one or more processors, cause the one or more processors to perform a method comprising:   displaying, on the display, a first user interface that provides information about generation, by the electronic device, of an avatar of a user of the electronic device;   while displaying the first user interface, receiving, via the one or more input devices, a request to initiate the generation of the avatar based on a digital representation of the user of the electronic device that is being captured by the camera of the electronic device;   in response to receiving the request to initiate the generation of the avatar, while capturing the digital representation of the user, displaying, on the display, a second user interface that includes:
 the digital representation of the user as it is being captured by the camera of the electronic device, and 
 a visual indication of a current status of evaluating the digital representation of the user of the electronic device that is being captured by the camera of the electronic device, wherein:
 the visual indication is overlaid on the digital representation of the user as the digital representation of the user is being evaluated, and 
 the visual indication moves from being overlaid on a first portion of the digital representation of the user corresponding to a first facial element to being overlaid on a second portion, different from the first portion, of the digital representation of the user corresponding to a second facial element, different from the first facial element, as the digital representation of the user is being evaluated; 
 
   after displaying the second user interface, displaying, on the display, a first proposed avatar based on the digital representation of the user; and   while displaying the first proposed avatar, providing the user with options for customizing the first proposed avatar.    
     
     
       44. The electronic device of claim 43, the method further comprising:
 after displaying the second user interface, concurrently displaying, on the display, the first proposed avatar and a second proposed avatar based on the digital representation of the user.   
     
     
       45. The electronic device of claim 44, the method further comprising:
 while concurrently displaying the first proposed avatar and the second proposed avatar on the display, receiving, via the one or more input devices, an input corresponding to a selection of a respective proposed avatar of the first and second proposed avatars; and   in response to receiving the input, entering a manual editing mode on the electronic device for manually editing the respective proposed avatar.   
     
     
       46. The electronic device of claim 43, wherein displaying the second user interface includes displaying dynamic feedback for guiding a position of the user that will improve an accuracy of the avatar that is being generated by the electronic device, and the feedback for guiding the position of the user comprises feedback for constraining the position of the user. 
     
     
       47. The electronic device of claim 46, wherein the feedback for constraining the position of the user comprises feedback for constraining a position of the user's head while the camera captures successive images of the user. 
     
     
       48. The electronic device of claim 43, the method further comprising:
 while displaying the first proposed avatar on the display, receiving, via the one or more input devices, a touch input from the user with respect to the first proposed avatar; and   in response to receiving the touch input, performing a corresponding action on the electronic device with respect to the first proposed avatar.   
     
     
       49. The electronic device of claim 48, wherein the touch input with respect to the first proposed avatar comprises touching of the first proposed avatar on the display. 
     
     
       50. The electronic device of claim 48, wherein the corresponding action comprises changing a display characteristic of the first proposed avatar on the display. 
     
     
       51. The electronic device of claim 50, wherein changing the display characteristic of the first proposed avatar on the display comprises enlarging the first proposed avatar on the display. 
     
     
       52. The electronic device of claim 43, wherein the visual indication of the current status indicates a respective portion of the user currently being evaluated. 
     
     
       53. The electronic device of claim 43, wherein the second user interface includes a latest candidate avatar of the user, overlaid on the digital representation of the user as it is being captured by the camera, based on a current progress of the evaluation of the digital representation of the user.