Deriving visual rhythm from video signals

Abstract

A method and a system for deriving visual rhythm from a video signal are described. A feature extraction module receives the video signal and extracts a two-dimensional feature from the video signal. A one-dimensional video feature computation module derives a one-dimensional feature from the extracted two-dimensional feature. A visual rhythm detector module detects a visual beat and a visual tempo from the one-dimensional feature.

1 - 26 . (canceled) 
     
     
         27 . A computer-implemented method, comprising:
 obtaining first information corresponding to a first signal, the first signal being associated with a first tempo;   identifying a second signal, the second signal being associated with a second tempo; and   generating, using one or more processors, a modification to the first signal that synchronizes the first signal to the second tempo.   
     
     
         28 . The method of  claim 27 , wherein:
 the first signal corresponds to an audio signal; and   the second signal corresponds to a video signal, the video signal having a plurality of frames;   
     
     
         29 . The method of  claim 28 , further comprising obtaining (i) first temporal positions corresponding to audio beats within the audio signal, and (ii) second temporal positions corresponding to visual beats within the video signal. 
     
     
         30 . The method of  claim 29 , wherein:
 the method further comprises:
 identifying (i) a first inter-onset interval for the audio signal based on the first temporal positions, and (ii) a second inter-onset interval for the video signal based on the second temporal positions; and 
 computing a ratio between the second inter-onset interval and the first inter-onset interval; and 
   the generating comprises modifying the first tempo of the first audio signal in accordance with the computed ratio.   
     
     
         31 . The method of  claim 30 , wherein the modifying comprises modifying a magnitude of the first inter-onset interval in accordance with the computed ratio, 
     
     
         32 . The method of  claim 29 , wherein:
 the method further comprises:
 determining a first tempo rate for the audio signal based on the first temporal positions; 
 determining a second tempo rate for the video signal based on the second temporal positions; and 
 computing a ratio of the second tempo rate to the first tempo rate; and 
   the generating comprises modifying the first tempo of the first audio signal in accordance with the computed ratio.   
     
     
         33 . The method of  claim 27 , wherein:
 the first video signal corresponds to a video signal, the video signal having a plurality of frames; and   the second signal corresponds to an audio signal.   
     
     
         34 . The method of  claim 33 , further comprising obtaining (i) first temporal positions corresponding to visual beats within the video signal, and (ii) second temporal positions corresponding to audio beats within the audio signal. 
     
     
         35 . The method of  claim 34 , wherein the generating comprising modifying a frame display rate of the video signal to synchronize the first and second temporal positions. 
     
     
         36 . The method of  claim 34 , further comprising:
 identifying a two-dimensional feature associated with at leas o of the frames of the video signal;   generating a one-dimensional feature based on the identified two-dimensional feature; and   determining the first tempo and at least one of the visual beats based on the one-dimensional feature.   
     
     
         37 . An apparatus, comprising:
 a storage device that stores a set of instructions; and   at least one processor coupled to the storage device, wherein the at least one processor, being operative with the set of instructions, is configured to:
 obtain first information corresponding to a first signal, the first signal being associated with a first tempo; 
 identify a second signal, the second signal being associated with a second tempo; and 
 generate, using one or more processors, a modification to the first signal that synchronizes the first signal to the second tempo. 
   
     
     
         38 . The apparatus of  claim 37 , wherein:
 the first signal corresponds to an audio signal; and   the second signal corresponds to a video signal, the video signal having a plurality of frames;   
     
     
         39 . The apparatus of  claim 38 , wherein the at least one processor is further configured to obtain (i) first temporal positions corresponding to audio beats within the audio signal, and (ii) second temporal positions corresponding to visual beats within the video signal. 
     
     
         40 . The apparatus of  claim 39 , wherein the at least one processor is further configured to:
 identify (i) a first inter-onset interval for the audio signal based on the first temporal positions, and (ii) a second inter-onset interval for the video signal based on the second temporal positions;   compute a ratio between the second inter-onset interval and the first inter-onset interval; and   modify the first tempo of the first audio signal in accordance with the computed ratio.   
     
     
         41 . The apparatus of  claim 40 , wherein the at least one processor is further configured to modify a magnitude of the first inter-onset interval in accordance with the computed ratio. 
     
     
         42 . The apparatus of  claim 39 , wherein the at least one processor is further configured to:
 determine a first tempo rate for the audio signal based on the first temporal positions;   determine a second tempo rate for the video signal based on the second temporal positions;   compute a ratio of the second tempo rate to the first tempo rate; and   modify the first tempo of the first audio signal in accordance with the computed ratio.   
     
     
         43 . The apparatus of  claim 37 , wherein:
 the first signal corresponds to a video signal, the video signal having a plurality of frames; and   the second signal corresponds to an audio signal.   
     
     
         44 . The apparatus of  claim 43 , wherein the at least one processor is further configured to:
 obtain (i) first temporal positions corresponding to visual beats within the video signal, and (ii) second temporal positions corresponding to audio beats within the audio signal; and   modify a frame display rate of the video signal to synchronize the first and second temporal positions.   
     
     
         45 . The apparatus of  claim 44 , wherein the at least one processor is further configured to:
 identify a two-dimensional feature associated with at least two of the frames of the video signal;   generate a one-dimensional feature based on the identified two-dimensional feature; and   determine the first tempo and at least one of the visual beats based on the one-dimensional feature.   
     
     
         46 . A tangible, non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform a method comprising:
 obtaining first information corresponding to a first signal, the first signal being associated with a first tempo;   identifying a second signal, the second signal being associated with a second tempo; and   generating a modification to the first signal that synchronizes the first signal to the second tempo.