Looping audio-visual file generation based on audio and video analysis

Abstract

In one general aspect, a method can include receiving a video loop portion included in a video file and receiving an audio loop portion included in an audio file. The method can include analyzing at least a portion of the audio file based on a musical characteristic and identifying a plurality of segment locations within the audio file based on the analyzing where the plurality of segment locations define a plurality of audio segments of the audio file. The method can also include modifying the video loop portion based on one of the plurality of segment locations in the audio file.

What is claimed is: 
     
         1 . A method comprising:
 analyzing at least a portion of an audio file based on an audio characteristic;   receiving a video loop portion included in a video file;   detecting video characteristics in the video loop portion including at least one of motion or color;   suggesting, based on the detected video characteristics, at least the audio file for combination with the video loop portion; and   automatically generating an audio-visual (AV) loop file that includes the video loop portion and at least a portion of the audio file.   
     
     
         2 . The method of  claim 1 , wherein:
 when the detected video characteristics include the motion and the video loop portion has a first level of motion, the audio file is suggested based on having a first beats per minute (BPM) rate corresponding to the first level of motion; and   when the detected video characteristics include the motion and the video loop portion has a second level of motion, the audio file is suggested based on having a second BPM rate corresponding to the second level of motion.   
     
     
         3 . The method of  claim 1 , wherein when the detected video characteristics include the color and the video loop portion has at least a threshold level of variation in color or contrast, the audio file is suggested based on having a beats per minute rate. 
     
     
         4 . The method of  claim 1 , further comprising identifying a plurality of segment locations within the audio file based on the analyzing, the plurality of segment locations defining a plurality of audio segments of the audio file. 
     
     
         5 . The method of  claim 4 , further comprising automatically modifying the video loop portion based on at least one of the plurality of segment locations. 
     
     
         6 . The method of  claim 5 , wherein automatically modifying the video loop portion comprises performing weighted algorithmic trimming in which a user's cuts in the video loop portion or the audio file are automatically trimmed so that a portion of the video loop portion or the audio file having a greatest relevance is included in the AV loop file. 
     
     
         7 . The method of  claim 6 , further comprising performing unbalanced trimming or unbalanced expansion as specified via a user preference. 
     
     
         8 . The method of  claim 7 , further comprising detecting an interaction with the video loop portion, wherein the interaction is taken into account in performing the unbalanced trimming or the unbalanced expansion. 
     
     
         9 . The method of  claim 5 , further comprising:
 presenting, in a user interface, an audio loop portion selector and segment location indicators representing at least some of the plurality of segment locations; and   receiving a user input made using the audio loop portion selector, the user input selecting a number of the presented segment location indicators;   wherein the AV loop file is generated based on the user input.   
     
     
         10 . The method of  claim 9 , further comprising limiting selection of the number of the presented segment locations using the audio loop portion selector by a time constraint. 
     
     
         11 . The method of  claim 10 , wherein the selection is limited so that (i) only a certain number of audio segments may be selected, or (ii) audio segments may only be selected if the audio segments have a collective duration that complies with the time constraint. 
     
     
         12 . The method of  claim 9 , wherein automatically generating the AV loop file comprises:
 decreasing the audio file in duration to the number of the presented segment location indicators selected using the audio loop portion selector; and   shifting the number of the presented segment location indicators in time from a first start time to a second start time, wherein an overall duration of the number of the presented segment location indicators is an integer multiple of a duration of one of the selected segment locations, wherein the selected number of the presented segment location indicators is a sliding window within the audio file.   
     
     
         13 . The method of  claim 1 , wherein the portion of the audio file being analyzed includes at least a first audio segment having a duration defined based on the audio characteristic, and wherein the video loop portion includes at least a first video segment and a second video segment, a duration of the video loop portion being different than the duration of the first audio segment, the method further comprising:
 identifying, based on the duration of the first audio segment, a first trim portion or a first expansion portion for the first video segment, and identifying a second trim portion or a second expansion portion for the second video segment; and   modifying the first video segment based on the first trim portion or the first expansion portion, and modifying the second video segment based on the second trim portion or the second expansion portion.   
     
     
         14 . The method of  claim 13 , further comprising, in response to the modifying of at least one of the first video segment or the second video segment being disproportionate, accordingly modifying at least one other of the first video segment or the second video segment. 
     
     
         15 . The method of  claim 14 , wherein identifying the first trim portion or the first expansion portion comprises identifying the first trim portion or the first expansion portion so as to have a different duration than the second trim portion or the second expansion portion. 
     
     
         16 . The method of  claim 14 , wherein identifying the first trim portion or the first expansion portion comprises taking into account a weight value. 
     
     
         17 . The method of  claim 1 , performed by software of an application executed on a mobile device, the application configured for generation of AV loop files from user-generated content, and for sharing of the AV loop files as multimedia content via a social network, within a social environment, within a social platform, or as social media content. 
     
     
         18 . The method of  claim 17 , wherein the application restricts the AV loop files being generated to a limited duration, and wherein an AV processor in the mobile device maintains the video loop portion and the portion of the audio file in sync while being capped by a time constraint. 
     
     
         19 . The method of  claim 17 , wherein the mobile device includes an AV player configured so that a preview in an edit view in a user interface matches a finalized version of the AV loop file, wherein a user experiments with segment locations by scrubbing and snapping to points using the user interface, an AV processor is configured to tune interactions to exceed a resolution available on a touchscreen of the mobile device, and wherein each movement is quantized to a musically relevant segment location. 
     
     
         20 . A non-transitory computer-readable medium having instructions for causing a processor to perform operations, the operations comprising:
 analyzing at least a portion of an audio file based on an audio characteristic;   receiving a video loop portion included in a video file;   detecting video characteristics in the video loop portion including at least one of motion or color;   suggesting, based on the detected video characteristics, at least the audio file for combination with the video loop portion; and   automatically generating an audio-visual (AV) loop file that includes the video loop portion and at least a portion of the audio file.