System and method for gesture-based image editing for self-portrait enhancement

Abstract

A computing device captures a live video of a user of the computing device and generates a user interface displaying the live video. The computing device detects a facial region of the user and tracks facial features within the facial region of the user. The computing device detects a presence of at least one finger in the live video with a threshold distance of a target facial feature and initiates a corresponding editing mode based on the target facial feature. The computing device edits an appearance of the target facial feature in the live video based on the editing mode and based on movement of the one or more fingers.

1 . A method implemented in a computing device, comprising:
 capturing a live video of a user of the computing device;   generating a user interface displaying the live video;   detecting a facial region of the user;   tracking facial features within the facial region of the user;   detecting a presence of at least one finger in the live video within a threshold distance of a target facial feature;   initiating a corresponding editing mode based on the target facial feature; and   editing an appearance of the target facial feature in the live video based on the editing mode and based on movement of the one or more fingers.   
     
     
         2 . The method of  claim 1 , wherein the corresponding editing mode is selected from one of a plurality of predefined editing modes, wherein the editing mode comprises one of:
 a first editing mode for reshaping an eye of the user;   a second editing mode for reshaping a chin of the user;   a third editing mode for modifying a width of the facial region;   a fourth editing mode for reshaping a nose of the user; and   a fifth editing mode for reshaping a mouth of the user.   
     
     
         3 . The method of  claim 2 , wherein editing the live video based on the first editing mode and based on the movement of the one or more fingers comprises reshaping the eye of the user based on the movement of one finger or based on the movement of an index finger and a thumb of the user. 
     
     
         4 . The method of  claim 2 , wherein editing the live video based on the second editing mode and based on the movement of the one or more fingers comprises reshaping the chin of the user based on the movement of one finger or based on the movement of an index finger and a thumb of the user. 
     
     
         5 . The method of  claim 2 , wherein editing the live video based on the third editing mode and based on the movement of the one or more fingers comprises reshaping a width of the facial region based on the movement of one finger or based on movement of an index finger and a thumb of the user. 
     
     
         6 . The method of  claim 2 , wherein editing the live video based on the fourth editing mode and based on the movement of the one or more fingers comprises reshaping the nose of the facial region based on the movement of one finger or based on the movement of an index finger and a thumb of the user. 
     
     
         7 . A system, comprising:
 a memory storing instructions;   a processor coupled to the memory and configured by the instructions to at least:   capture a live video of a user;   generate a user interface displaying the live video;   detect a facial region of the user;   track facial features within the facial region of the user;   detect a presence of at least one finger in the live video within a threshold distance of a target facial feature;   initiate a corresponding editing mode based on the target facial feature; and   edit an appearance of the target facial feature in the live video based on the editing mode and based on movement of the one or more fingers.   
     
     
         8 . The system of  claim 7 , wherein the corresponding editing mode is selected from one of a plurality of predefined editing modes, wherein the editing mode comprises one of:
 a first editing mode for reshaping an eye of the user;   a second editing mode for reshaping a chin of the user;   a third editing mode for modifying a width of the facial region;   a fourth editing mode for reshaping a nose of the user; and   a fifth editing mode for reshaping a mouth of the user.   
     
     
         9 . The system of  claim 8 , wherein the processor is configured to edit the live video based on the first editing mode and based on the movement of the one or more fingers by reshaping the eye of the user based on the movement of one finger or based on the movement of an index finger and a thumb of the user. 
     
     
         10 . The system of  claim 8 , wherein the processor is configured to edit the live video based on the second editing mode and based on the movement of the one or more fingers by reshaping the chin of the user based on the movement of one finger or based on the movement of an index finger and a thumb of the user. 
     
     
         11 . The system of  claim 8 , wherein the processor is configured to edit the live video based on the third editing mode and based on the movement of the one or more fingers by reshaping a width of the facial region based on the movement of one finger or based on movement of an index finger and a thumb of the user. 
     
     
         12 . The system of  claim 8 , wherein the processor is configured to edit the live video based on the fourth editing mode and based on the movement of the one or more fingers by reshaping the nose of the facial region based on the movement of one finger or based on the movement of an index finger and a thumb of the user. 
     
     
         13 . A non-transitory computer-readable storage medium storing instructions to be implemented by a computing device having a processor, wherein the instructions, when executed by the processor, cause the computing device to at least:
 capture a live video of a user of the computing device;   generate a user interface displaying the live video;   detect a facial region of the user;   track facial features within the facial region of the user;   detect a presence of at least one finger in the live video within a threshold distance of a target facial feature;   initiate a corresponding editing mode based on the target facial feature; and   edit an appearance of the target facial feature in the live video based on the editing mode and based on movement of the one or more fingers.   
     
     
         14 . The non-transitory computer-readable storage medium of  claim 13 , wherein the corresponding editing mode is selected from one of a plurality of predefined editing modes, wherein the editing mode comprises one of:
 a first editing mode for reshaping an eye of the user;   a second editing mode for reshaping a chin of the user;   a third editing mode for modifying a width of the facial region;   a fourth editing mode for reshaping a nose of the user; and   a fifth editing mode for reshaping a mouth of the user.   
     
     
         15 . The non-transitory computer-readable storage medium of  claim 14 , wherein the processor is configured to edit the live video based on the first editing mode and based on the movement of the one or more fingers by reshaping the eye of the user based on the movement of one finger or based on the movement of an index finger and a thumb of the user. 
     
     
         16 . The non-transitory computer-readable storage medium of  claim 14 , wherein the processor is configured to edit the live video based on the second editing mode and based on the movement of the one or more fingers by reshaping the chin of the user based on the movement of one finger or based on the movement of an index finger and a thumb of the user. 
     
     
         17 . The non-transitory computer-readable storage medium of  claim 14 , wherein the processor is configured to edit the live video based on the third editing mode and based on the movement of the one or more fingers by reshaping a width of the facial region based on the movement of one finger or based on movement of an index finger and a thumb of the user. 
     
     
         18 . The non-transitory computer-readable storage medium of  claim 14 , wherein the processor is configured to edit the live video based on the fourth editing mode and based on the movement of the one or more fingers by reshaping the nose of the facial region based on the movement of one finger or based on the movement of an index finger and a thumb of the user.