Method and device for generating and detecting fingerprints for synchronizing audio and video

Abstract

This invention relates to a device and a method of generating a first and a second fingerprint ( 102,104 ) usable for synchronisation of at least two signals ( 101,103 ) and corresponding method and device for synchronising two or more signals. A fingerprint pair is generated on the basis of a segment of a first signal e.g. an audio signal and of a segment of a second signal e.g. a video signal at each synchronisation time point. The generated fingerprint pair(s) are stored in a database ( 203 ) and communicated or distributed to a synchronisation device ( 303 ). During synchronisation, fingerprint(s) of the audio signal and fingerprint(s) of the video signal to be synchronised are generated and matched against fingerprints in the database. When a match is found, the fingerprints also determine the synchronisation time point, which is used to synchronise the two signals. In this way, a simple, reliable and efficient way of synchronising at least two signals is obtained. Further, this is enabled without modifying either the first and second signal (or subsequent signals). The signals may even be distorted or changed to some extent while still enabling synchronisation.

1 . A method comprising:
 accessing a first fingerprint associated with a first signal;   accessing a second fingerprint associated with a second signal, the first fingerprint and the second fingerprint forming a fingerprint pair that includes information to identify a first point within the first signal and a second point within the second signal at which the first and second signals are synchronized relative to each other;   using a processor, processing a search configured to find a first match to the first fingerprint among first fingerprints generated from the first signal and find a second match to the second fingerprint among second fingerprints generated from the second signal, the first match and the second match forming a match pair that indicates an offset by which the first point and the second point differ; and   compensating for the offset by shifting playback of at least one of the first signal or the second signal relative to each other.   
     
     
         2 . The method of  claim 1 , wherein:
 the first point is a first time point within the first signal;   the second point is a second time point within the second signal at which the first and second signals are synchronized relative to each other; and   the fingerprint pair formed by the first fingerprint and the second fingerprint includes information to identify the first time point within the first signal and the second time point within the second signal.   
     
     
         3 . The method of  claim 2 , wherein:
 the first fingerprint is generated from a first segment that corresponds to the first time point within the first signal;   the second fingerprint is generated from a second segment that corresponds to the second time point within the second signal; and   the fingerprint pair formed by the first and second fingerprints includes information to identify the first and second segments at which the first and second signals are synchronized relative to each other.   
     
     
         4 . The method of  claim 2 , wherein:
 the first and second time points share a reference timeline in common; and   the fingerprint pair formed by the first and second fingerprints includes information to identify the first and second points on the reference timeline at which the first and second signals are synchronized relative to each other.   
     
     
         5 . The method of  claim 2 , wherein:
 the first time point and the second time point differ by the offset; and   the compensating for the offset shifts the playback of at least one of the first signal or the second signal such that the first and second signals are played without the offset at the first and second time points.   
     
     
         6 . The method of  claim 1 , wherein:
 the first point is a first location within the first signal;   the second point is a second location within the second signal at which the first and second signals are synchronized relative to each other; and   the fingerprint pair formed by the first fingerprint and the second fingerprint includes information to identify the first location within the first signal and the second location within the second signal.   
     
     
         7 . The method of  claim 6 , wherein:
 the first fingerprint is generated from a first segment that corresponds to the first location within the first signal;   the second fingerprint is generated from a second segment that corresponds to the second location within the second signal; and   the fingerprint pair formed by the first and second fingerprints includes information to identify the first and second segments at which the first and second signals are synchronized relative to each other.   
     
     
         8 . The method of  claim 6 , wherein:
 the first location and the second location differ by the offset; and   the compensating for the offset shifts the playback of at least one of the first signal or the second signal such that the first and second signals are played without the offset at the first and second locations.   
     
     
         9 . The method of  claim 1 , wherein:
 the processing of the search includes providing the first match to the first fingerprint and the second match to the second fingerprint to a device in response to the search.   
     
     
         10 . The method of  claim 9  further comprising:
 generating the first fingerprints from the first signal and storing the generated first fingerprints in a database; and 
 generating the second fingerprints from the second signal and storing the generated second fingerprints in the database. 
 
     
     
         11 . The method of  claim 9 , wherein:
 the processing of the search includes providing the first fingerprints and the second fingerprints to the device, the first match being among the first fingerprints and the second match being among the second fingerprints.   
     
     
         12 . The method of  claim 1 , wherein:
 the processing of the search includes receiving the first match to the first fingerprint and the second match to the second fingerprint from a server in response to the search.   
     
     
         13 . The method of  claim 12 , wherein:
 the accessing of the first fingerprint includes generating the first fingerprint from a first version of the first signal; and   the accessing of the second fingerprint includes generating the second fingerprint from a second version of the second signal.   
     
     
         14 . The method of  claim 12 , wherein:
 the processing of the search includes receiving the first fingerprints and the second fingerprints from the server, the first match being among the first fingerprint and the second match being among the second fingerprints.   
     
     
         15 . The method of  claim 1 , wherein:
 the first signal is an audio signal; and   the second signal is a video signal.   
     
     
         16 . A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
 accessing a first fingerprint associated with a first signal;   accessing a second fingerprint associated with a second signal, the first fingerprint and the second fingerprint forming a fingerprint pair that includes information to identify a first point within the first signal and a second point within the second signal at which the first and second signals are synchronized relative to each other;   processing a search configured to find a first match to the first fingerprint among first fingerprints generated from the first signal and find a second match to the second fingerprint among second fingerprints generated from the second signal, the first match and the second match forming a match pair that indicates an offset by which the first point and the second point differ; and   compensating for the offset by shifting playback of at least one of the first signal or the second signal relative to each other.   
     
     
         17 . The non-transitory machine-readable storage medium of  claim 16 , wherein:
 the first point is a first location within the first signal;   the second point is a second location within the second signal at which the first and second signals are synchronized relative to each other; and   the fingerprint pair formed by the first fingerprint and the second fingerprint includes information to identify the first location within the first signal and the second location within the second signal.   
     
     
         18 . A system comprising:
 a fingerprint module configured to:
 access a first fingerprint associated with a first signal; and 
 access a second fingerprint associated with a second signal, the first fingerprint and the second fingerprint forming a fingerprint pair that includes information to identify a first point within the first signal and a second point within the second signal at which the first and second signals are synchronized relative to each other; 
   a processor configured to process a search configured to find a first match to the first fingerprint among first fingerprints generated from the first signal and find a second match to the second fingerprint among second fingerprints generated from the second signal, the first match in the second match forming a match pair that indicates an offset by which the first point and the second point differ; and   a synchronization module configured to compensate for the offset by shifting playback of at least one of the first signal or the second signal relative to each other.   
     
     
         19 . The system of  claim 18 , wherein:
 the first point is a first location within the first signal;   the second point is a second location within the second signal at which the first and second signals are synchronized relative to each other; and   the fingerprint pair formed by the first fingerprint and the second fingerprint includes information to identify the first location within the first signal and the second location within the second signal.   
     
     
         20 . The system of  claim 19 , wherein:
 the fingerprint module includes a fingerprint generator configured to:
 generate the first fingerprint from a first segment that corresponds to the first location within the first signal; and 
 generate the second fingerprint from a second segment that corresponds to the second location within the second signal; and 
   the fingerprint pair formed by the first and second fingerprints includes information to identify the first and second segments at which the first and second signals are synchronized relative to each other.