Image processing method and apparatus for augmented reality, electronic device, and storage medium

Abstract

An image processing method and apparatus for augmented reality, an electronic device and a storage medium, including: acquiring a target image in response to an image acquiring instruction triggered by a user, where the target image includes a target object; acquiring an augmented reality model of the target object, and outputting the augmented reality model in combination with the target object; acquiring target audio data selected by the user, and determining an audio feature with temporal regularity according to the target audio data; and driving the augmented reality model according to the audio feature and a playing progress of the target audio data when outputting the target audio data.

What is claimed is: 
     
       1. An image processing method for augmented reality, comprising:
 acquiring a target image in response to an image acquiring instruction triggered by a user, wherein the target image comprises a target object; 
 acquiring an augmented reality model of the target object, and outputting the augmented reality model in combination with the target object, wherein the augmented reality model comprises a plurality of model units, and the plurality of model units are spliced to form the augmented reality model; 
 acquiring target audio data selected by the user; 
 determining an audio feature with temporal regularity according to the target audio data; and 
 driving the augmented reality model according to the audio feature and a playing progress of the target audio data when outputting the target audio data, 
 wherein driving the augmented reality model according to the audio feature and the playing progress of the target audio data comprises: 
 driving at least part of the model units in the augmented reality model according to the audio feature and the playing progress of the target audio data. 
 
     
     
       2. The method according to  claim 1 , wherein determining the audio feature with temporal regularity according to the target audio data comprises:
 performing audio feature detection on the target audio data to obtain the audio feature with temporal regularity of the target audio data, wherein the audio feature comprises a combination of one or more of stress, downbeat or beat. 
 
     
     
       3. The method according to  claim 1 , wherein driving at least part of the model units in the augmented reality model according to the audio feature and the playing progress of the target audio data comprises:
 determining target time and a target amplitude of a morphological change of the model unit according to the audio feature with temporal regularity; and 
 driving the model unit in the augmented reality model according to the target amplitude if the playing progress of the target audio data is the target time. 
 
     
     
       4. The method according to  claim 3 , wherein driving the model unit in the augmented reality model according to the target amplitude comprises:
 driving a preset model unit in the augmented reality model to perform a bulge action; or, 
 driving a plurality of model units in the augmented reality model to perform color changing; or 
 driving a plurality of model units in the augmented reality model to perform transparency changing. 
 
     
     
       5. The method according to  claim 1 , wherein after acquiring the target audio data selected by the user, the method further comprises:
 acquiring an emotional feature of the target audio data; 
 determining a first deformation feature of the augmented reality model according to the emotional feature, wherein the first deformation feature is used to drive a shape of the augmented reality model to be consistent with emotion expressed by the emotional feature; 
 driving the augmented reality model according to the first deformation feature when outputting the target audio data. 
 
     
     
       6. The method according to  claim 2 , wherein after acquiring the target audio data selected by the user, the method further comprises:
 acquiring an emotional feature of the target audio data; 
 determining a first deformation feature of the augmented reality model according to the emotional feature, wherein the first deformation feature is used to drive a shape of the augmented reality model to be consistent with emotion expressed by the emotional feature; 
 driving the augmented reality model according to the first deformation feature when outputting the target audio data. 
 
     
     
       7. The method according to  claim 1 , wherein after acquiring the target audio data selected by the user, the method further comprises:
 acquiring a body movement or an expression of the user; 
 determining a second deformation feature of the augmented reality model according to the body movement or the expression, wherein the second deformation feature is used to drive a shape of the augmented reality model to be consistent with the body movement or the expression; 
 driving the augmented reality model according to the second deformation feature when outputting the target audio data. 
 
     
     
       8. The method according to  claim 2 , wherein after acquiring the target audio data selected by the user, the method further comprises:
 acquiring a body movement or an expression of the user; 
 determining a second deformation feature of the augmented reality model according to the body movement or the expression, wherein the second deformation feature is used to drive a shape of the augmented reality model to be consistent with the body movement or the expression; 
 driving the augmented reality model according to the second deformation feature when outputting the target audio data. 
 
     
     
       9. An electronic device, comprising:
 one or more processors; 
 a storage, configured to store one or more programs, 
 the one or more programs, when executed by the one or more processors, cause the one or more processors to: 
 acquire a target image in response to an image acquiring instruction triggered by a user, wherein the target image comprises a target object; 
 acquire an augmented reality model of the target object and output the augmented reality model in combination with the target object, wherein the augmented reality model comprises a plurality of model units, and the plurality of model units are spliced to form the augmented reality model; 
 acquire target audio data selected by the user; 
 determine an audio feature with temporal regularity according to the target audio data; and 
 drive the augmented reality model according to the audio feature and a playing progress of the target audio data when outputting the target audio data; 
 wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to: 
 drive at least part of the model units in the augmented reality model according to the audio feature and the playing progress of the target audio data. 
 
     
     
       10. The electronic device according to  claim 9 , wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to:
 perform audio feature detection on the target audio data to obtain the audio feature with temporal regularity of the target audio data, wherein the audio feature comprises a combination of one or more of stress, downbeat or beat. 
 
     
     
       11. The electronic device according to  claim 9 , wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to:
 determine target time and a target amplitude of a morphological change of the model unit according to the audio feature with temporal regularity; 
 drive the model unit in the augmented reality model according to the target amplitude if the playing progress of the target audio data is the target time. 
 
     
     
       12. The electronic device according to  claim 11 , wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to:
 drive a preset model unit in the augmented reality model to perform a bulge action; or, 
 drive a plurality of model units in the augmented reality model to perform color changing; or 
 drive a plurality of model units in the augmented reality model to perform drive a plurality of model units in the augmented reality model to perform transparency changing. 
 
     
     
       13. The electronic device according to  claim 9 , wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to:
 acquire an emotional feature of the target audio data; 
 determine a first deformation feature of the augmented reality model according to the emotional feature, wherein the first deformation feature is used to drive a shape of the augmented reality model to be consistent with emotion expressed by the emotional feature; 
 drive the augmented reality model according to the first deformation feature when outputting the target audio data. 
 
     
     
       14. The electronic device according to  claim 10 , wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to:
 acquire an emotional feature of the target audio data; 
 determine a first deformation feature of the augmented reality model according to the emotional feature, wherein the first deformation feature is used to drive a shape of the augmented reality model to be consistent with emotion expressed by the emotional feature; 
 drive the augmented reality model according to the first deformation feature when outputting the target audio data. 
 
     
     
       15. The electronic device according to  claim 9 , wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to:
 acquire a body movement or an expression of the user; 
 determine a second deformation feature of the augmented reality model according to the body movement or the expression, wherein the second deformation feature is used to drive a shape of the augmented reality model to be consistent with the body movement or the expression; 
 drive the augmented reality model according to the second deformation feature when outputting the target audio data. 
 
     
     
       16. A non-transitory storage medium having computer executable instructions thereon, the computer executable instructions, when executed by a computer processor, cause the computer processor to execute the image processing method for augmented reality according to  claim 1 .