Creating videos with facial expressions

Abstract

The present disclosure relates to creating videos. A mobile device creates a graphic user interface to capture by the camera of the device multiple photographic facial images of a user for respective multiple facial expressions of a character in the video. Using the multiple photographic facial images, the device modifies stored character images by matching facial features of the character to facial features of the user for the multiple facial expressions of the character in the video and creates the video based on the modified character images. The facial expression of the user is used to influence the facial expression of the character. This method enables replacement of certain visual style elements with a given user's own style elements.

1 . A method for creating a video on a mobile device that comprises a camera, the method comprising:
 creating a graphic user interface on the mobile device to capture by the camera multiple photographic facial images of a user for respective multiple facial expressions of a character in the video;   using the multiple photographic facial images to modify stored character images by:
 matching facial features of the character, comprising control points, to facial features of the user for the multiple facial expressions of the character in the video, and 
 interpolating between the control points associated with a first facial expression of the multiple facial expressions and control points associated with a second facial expression of the multiple facial expressions; and 
   creating the video based on the modified character images.   
     
     
         2 . A method for creating a video including a character on a mobile device that comprises a camera, the method comprising:
 (a) creating a graphic user interface on the mobile device to capture by the camera multiple photographic facial images of a user for respective multiple facial expressions;   (b) extracting a user facial feature, comprising control points, from each of the multiple photographic facial images;   (c) storing associated with a respective facial expression identifier the user facial feature from each of the multiple photographic facial images;   (d) selecting one of the multiple user facial features based on a first facial expression identifier associated with a first frame of the video;   (e) determining a transformation that transforms a reference facial feature, comprising control points, associated with the first facial expression identifier into an approximation or representation of the selected one of the multiple user facial features;   (f) modifying, based on the transformation, a reference facial image of the character associated with the first facial expression identifier and the reference facial feature; and   (g) creating the first frame of the video based on the modified reference facial image;   for a second facial expression identifier associated with a second frame of the video, repeating steps (d) to (g) to create the second frame of the video; and   interpolating between control points of the reference facial feature associated with the first frame of the video and control points of the reference facial feature associated with the second frame of the video to create an output frame between the first frame and the second frame.   
     
     
         5 . The method of any one of preceding claims, wherein the graphic user interface comprises the reference facial image of the character. 
     
     
         6 . The method of any one of preceding claims, wherein the graphic user interface comprises a live view of each of the multiple photographic facial images. 
     
     
         7 . The method of  claim 6 , wherein the live view is positioned next to the camera. 
     
     
         8 . The method of  claim 7 , wherein the live view is positioned next to the reference facial image of the character. 
     
     
         9 . The method of  claim 6 , further comprising superimposing the live view on the reference facial image of the character. 
     
     
         10 . The method of any one of preceding claims, further comprising selecting the character from a plurality of characters in the video. 
     
     
         11 . The method of any one of preceding claims, further comprising recording audio data associated with the user facial feature. 
     
     
         12 . A computer software product, including machine-readable instructions, when executed by a processor of a mobile device, causes the processor to perform any one of preceding methods. 
     
     
         13 . A mobile device for creating a video including a character, the mobile device comprising
 a camera;   a display; and   a processor, the processor configured to   (a) create a graphic user interface on the display of the mobile device to capture by the camera multiple photographic facial images of a user for respective multiple facial expressions;   (b) extract a user facial feature, comprising control points, from each of the multiple photographic facial images;   (c) store associated with a respective facial expression identifier the user facial feature from each of the multiple photographic facial images;   (d) select one of the multiple user facial features based on a first facial expression identifier associated with a first frame of the video;   (e) determine a transformation that transforms a reference facial feature, comprising control points, associated with the first facial expression identifier into an approximation or representation of the selected one of the multiple user facial features;   (f) modify, based on the transformation, a reference facial image of the character associated with the first facial expression identifier and the reference facial feature;   (g) create the first frame of the video based on the modified reference facial image; and   (h) present, on the display, the first frame of the video in the graphic user interface.   for a second facial expression identifier associated with a second frame of the video, repeat steps (d) to (g) to create the second frame of the video; and   interpolate between control points of the reference facial feature associated with the first frame of the video and control points of the reference facial feature associated with the second frame of the video to create an output frame between the first frame and the second frame.