Audio and Video Synchronization

Abstract

Concepts and technologies disclosed herein are directed to audio and video synchronization. According to one aspect disclosed herein, an audio-video (“AV”) synchronization system can simultaneously capture samples of a pre-encode media stream and a post-encode media stream. The pre-encode media stream can include AV content prior to being encoded. The post-encode media stream can include the AV content after being encoded. The AV synchronization system can align a pre-encode video component of the pre-encode media stream with a post-encode video component and can determine a video offset therebetween. The AV synchronization system can align a pre-encode audio component of the pre-encode media stream with a post-encode audio component of the post-encode media stream and can determine an audio offset therebetween. The AV synchronization system can then compare the video offset and the audio offset to determine if the post-encode media stream is synchronized with the pre-encode media stream.

1 . A method comprising:
 demultiplexing, by an audio video (“AV”) synchronization system comprising a processor, a pre-encode AV content into a pre-encode audio and pre-encode video;   demultiplexing, by the AV synchronization system, a post-encode AV content into post-encode audio and post-encode video;   creating, by the AV synchronization system, a first video pipe;   outputting, by the AV synchronization system, the pre-encode video on the first video pipe towards a non-annotated AV processing algorithm module;   creating, by the AV synchronization system, a second video pipe;   outputting, by the AV synchronization system, the post-encode video on the second video pipe towards the non-annotated AV processing algorithm module;   generating, by the non-annotated AV processing algorithm module executed by the processor of the AV synchronization system, a plurality of thumbnail images from the pre-encode video and the post-encode video;   determining, by the non-annotated AV processing algorithm module, a plurality of search ranges for an iterative search process;   determining, by the non-annotated AV processing algorithm module, using the plurality of search ranges during the iterative search process, a first alignment point between the pre-encode video and the post-encode video;   comparing, by the non-annotated AV processing algorithm module, the plurality of thumbnail images to determine a plurality of distance values;   determining, by the non-annotated AV processing algorithm module, a second alignment point between the pre-encode video and the post-encode video, wherein the second alignment point is where a distance value of the plurality of distance values is minimized;   determining, by the non-annotated AV processing algorithm module, based upon the first alignment point and the second alignment point, a video offset between the pre-encode video and the post-encode video; and   outputting, by the non-annotated AV processing algorithm module, the video offset to an AV synchronization module.   
     
     
         2 . The method of  claim 1 , further comprising:
 creating, by the AV synchronization system, a first audio pipe;   outputting, by the AV synchronization system, the pre-encode audio on the first audio pipe towards the AV synchronization module;   creating, by the AV synchronization system, a second audio pipe;   outputting, by the AV synchronization system, the post-encode audio on the second audio pipe towards the AV synchronization module;   aligning, by the AV synchronization module executed by the AV synchronization system, the pre-encode audio with the post-encode audio;   determining, by the AV synchronization module, an audio offset between the pre-encode audio and the post-encode audio; and   comparing, by the AV synchronization system, the video offset and the audio offset to determine if the post-encode AV content is synchronized with the pre-encode AV content.   
     
     
         3 . The method of  claim 2 , wherein aligning, by the AV synchronization module, the pre-encode audio with the post-encode audio:
 dividing, by the AV synchronization module, the pre-encode audio and the post-encode audio into a plurality of time slices, wherein each time slice of the plurality of time slices is associated with a corresponding video frame;   generating, by the AV synchronization module, acoustic fingerprints based upon the plurality of time slices; and   performing, by the AV synchronization module, fingerprint matching using the acoustic fingerprints and determining the audio offset therefrom.   
     
     
         4 . The method of  claim 3 , further comprising outputting, by the AV synchronization system, an AV synchronization evaluation result. 
     
     
         5 . The method of  claim 4 , wherein the AV synchronization evaluation result comprises an indication that the pre-encode AV content and the post-encode AV content are not synchronized. 
     
     
         6 . The method of  claim 4 , wherein the AV synchronization evaluation result comprises an indication that the pre-encode AV content and the post-encode AV content are synchronized. 
     
     
         7 . The method of  claim 1 , wherein the plurality of thumbnail images are bi-tonal images. 
     
     
         8 . A audio video (“AV”) synchronization system comprising:
 a processor; and 
 a memory that stores a plurality of modules comprising instructions that, when executed by the processor, cause the processor to perform operations comprising
 demultiplexing a pre-encode AV content into a pre-encode audio and pre-encode video, 
 demultiplexing a post-encode AV content into post-encode audio and post-encode video, 
 creating a first video pipe, 
 outputting the pre-encode video on the first video pipe towards a non-annotated AV processing algorithm module of the plurality of modules, 
 creating a second video pipe, 
 outputting the post-encode video on the second video pipe towards the non-annotated AV processing algorithm module, 
 generating, by the non-annotated AV processing algorithm module, a plurality of thumbnail images from the pre-encode video and the post-encode video, 
 determining, by the non-annotated AV processing algorithm module, a plurality of search ranges for an iterative search process, 
 determining, by the non-annotated AV processing algorithm module, using the plurality of search ranges during the iterative search process, a first alignment point between the pre-encode video and the post-encode video, 
 comparing, by the non-annotated AV processing algorithm module, the plurality of thumbnail images to determine a plurality of distance values, 
 determining, by the non-annotated AV processing algorithm module, a second alignment point between the pre-encode video and the post-encode video, wherein the second alignment point is where a distance value of the plurality of distance values is minimized, 
 determining, by the non-annotated AV processing algorithm module, based upon the first alignment point and the second alignment point, a video offset between the pre-encode video and the post-encode video, and 
 outputting the video offset to an AV synchronization module. 
 
 
     
     
         9 . The AV synchronization system of  claim 8 , wherein the operations further comprise:
 creating a first audio pipe;   outputting the pre-encode audio on the first audio pipe towards the AV synchronization module;   creating a second audio pipe;   outputting the post-encode audio on the second audio pipe towards the AV synchronization module;   aligning, by the AV synchronization module, the pre-encode audio with the post-encode audio;   determining, by the AV synchronization module, an audio offset between the pre-encode audio and the post-encode audio; and   comparing the video offset and the audio offset to determine if the post-encode AV content is synchronized with the pre-encode AV content.   
     
     
         10 . The AV synchronization system of  claim 9 , wherein aligning, by the AV synchronization module, the pre-encode audio with the post-encode audio:
 dividing, by the AV synchronization module, the pre-encode audio and the post-encode audio into a plurality of time slices, wherein each time slice of the plurality of time slices is associated with a corresponding video frame;   generating, by the AV synchronization module, acoustic fingerprints based upon the plurality of time slices; and   performing, by the AV synchronization module, fingerprint matching using the acoustic fingerprints and determining the audio offset therefrom.   
     
     
         11 . The AV synchronization system of  claim 10 , wherein the operations comprise outputting an AV synchronization evaluation result. 
     
     
         12 . The AV synchronization system of  claim 11 , wherein the AV synchronization evaluation result comprises an indication that the pre-encode AV content and the post-encode AV content are not synchronized. 
     
     
         13 . The AV synchronization system of  claim 11 , wherein the AV synchronization evaluation result comprises an indication that the pre-encode AV content and the post-encode AV content are synchronized. 
     
     
         14 . The AV synchronization system of  claim 8 , wherein the plurality of thumbnail images are bi-tonal images. 
     
     
         15 . A computer-readable storage medium having a plurality of modules comprising instructions stored thereon that, when executed by a processor of an audio video (“AV”) system, cause the processor to perform operations comprising:
 demultiplexing a pre-encode AV content into a pre-encode audio and pre-encode video; 
 demultiplexing a post-encode AV content into post-encode audio and post-encode video; 
 creating a first video pipe; 
 outputting the pre-encode video on the first video pipe towards a non-annotated AV processing algorithm module of the plurality of modules; 
 creating a second video pipe; 
 outputting the post-encode video on the second video pipe towards the non-annotated AV processing algorithm module; 
 generating, by the non-annotated AV processing algorithm module, a plurality of thumbnail images from the pre-encode video and the post-encode video; 
 determining, by the non-annotated AV processing algorithm module, a plurality of search ranges for an iterative search process; 
 determining, by the non-annotated AV processing algorithm module, using the plurality of search ranges during the iterative search process, a first alignment point between the pre-encode video and the post-encode video; 
 comparing, by the non-annotated AV processing algorithm module, the plurality of thumbnail images to determine a plurality of distance values; 
 determining, by the non-annotated AV processing algorithm module, a second alignment point between the pre-encode video and the post-encode video, wherein the second alignment point is where a distance value of the plurality of distance values is minimized; 
 determining, by the non-annotated AV processing algorithm module, based upon the first alignment point and the second alignment point, a video offset between the pre-encode video and the post-encode video; and 
 outputting the video offset to an AV synchronization module. 
 
     
     
         16 . The computer-readable storage medium of  claim 15 , wherein the operations further comprise:
 creating a first audio pipe;   outputting the pre-encode audio on the first audio pipe towards the AV synchronization module;   creating a second audio pipe;   outputting the post-encode audio on the second audio pipe towards the AV synchronization module;   aligning, by the AV synchronization module, the pre-encode audio with the post-encode audio;   determining, by the AV synchronization module, an audio offset between the pre-encode audio and the post-encode audio; and   comparing the video offset and the audio offset to determine if the post-encode AV content is synchronized with the pre-encode AV content.   
     
     
         17 . The computer-readable storage medium of  claim 16 , wherein aligning, by the AV synchronization module, the pre-encode audio with the post-encode audio:
 dividing, by the AV synchronization module, the pre-encode audio and the post-encode audio into a plurality of time slices, wherein each time slice of the plurality of time slices is associated with a corresponding video frame;   generating, by the AV synchronization module, acoustic fingerprints based upon the plurality of time slices; and   performing, by the AV synchronization module, fingerprint matching using the acoustic fingerprints and determining the audio offset therefrom.   
     
     
         18 . The computer-readable storage medium of  claim 17 , wherein the operations comprise outputting an AV synchronization evaluation result. 
     
     
         19 . The computer-readable storage medium of  claim 18 , wherein the AV synchronization evaluation result comprises an indication that the pre-encode AV content and the post-encode AV content are not synchronized. 
     
     
         20 . The computer-readable storage medium of  claim 18 , wherein the AV synchronization evaluation result comprises an indication that the pre-encode AV content and the post-encode AV content are synchronized.