System and method for converting audio-to-text with delay

Abstract

Described herein is a system and method for generating text caption information for an audio-video (AV) signal, the systems and method comprising: receiving an AV signal; extracting audio from the AV signal to form an extracted audio signal; time stamping both the extracted audio signal and the received AV signal; partitioning the extracted audio signal into a first predetermined duration segment of extracted audio signal; generating text captions from the partitioned extracted audio signal over a first duration, and converting the same to a video text signal, with the same time stamp as the extracted audio signal and received AV signal; delaying the received AV signal by an amount of time substantially similar to the first duration; combining the time stamped video text signal and the delayed time stamped received AV signal based on the time stamps; and outputting the combined time stamped video text signal and the time stamped received AV signal to a display.

What is claimed is: 
     
         1 . A method for generating text caption information for an audio-video (AV) signal, the method comprising:
 receiving an AV signal;   extracting audio from the AV signal to form an extracted audio signal;   time stamping both the extracted audio signal and the received AV signal;   partitioning the extracted audio signal into a first predetermined duration segment of extracted audio signal;   generating text captions from the partitioned extracted audio signal over a first duration, and converting the same to a video text signal, with the same time stamp as the extracted audio signal and received AV signal;   delaying the received AV signal by an amount of time substantially similar to the first duration;   combining the time stamped video text signal and the delayed time stamped received AV signal based on the time stamps; and   outputting the combined time stamped video text signal and the time stamped received AV signal to a display.   
     
     
         2 . The method according to  claim 1 , wherein the step of generating text captions from the partitioned extracted audio signal over a first duration further comprises:
 comparing the generated text captions with a list of text obtained by a source of the AV signal to improve accuracy of the generated text captions.   
     
     
         3 . The method according to  claim 2 , wherein the list of text obtained by the source of the AV signal comprises text associated with the subject matter of the AV signal. 
     
     
         4 . The method according to  claim 1 , wherein the step of generating text captions from the partitioned extracted audio signal over a first duration further comprises:
 obtaining metadata from the AV signal;   generating a list of text that substantially matches the subject matter of the AV signal based on the obtained metadata;   comparing the generated text captions with the generated list of text based on the obtained metadata to improve accuracy of the generated text captions.   
     
     
         5 . The method according to  claim 1 , wherein the step of generating text captions from the partitioned extracted audio signal over a first duration further comprises:
 using artificial intelligence programming techniques to develop a list of text that substantially matches the subject matter of the AV signal based on the obtained metadata;   comparing the generated text captions with the AI developed list of text to improve accuracy of the generated text captions.   
     
     
         6 . The method according to  claim 5 , wherein the AI techniques comprise:
 Recurrent Neural Networks that are trained to suppress non-voice audio resulting in significantly improved voice signal-to-noise ratio (SNR) and clarity.   
     
     
         7 . A system for generating text caption information for an audio-video (AV) signal system, comprising:
 an audio-video (AV) signal receiver;   at least one processor that is part of the AV signal receiver;   a memory operatively connected with the at least one processor, wherein the memory stores computer-executable instructions that, when executed by the at least one processor, causes the at least one processor to execute a method that comprises:
 receiving an AV signal at the AV signal receiver; 
 extracting audio from the AV signal to form an extracted audio signal; 
 time stamping both the extracted audio signal and the received AV signal; 
 partitioning the extracted audio signal into a first predetermined duration segment of extracted audio signal; 
 generating text captions from the partitioned extracted audio signal over a first duration, and converting the same to a video text signal, with the same time stamp as the extracted audio signal and received AV signal; 
 delaying the received AV signal by an amount of time substantially similar to the first duration; 
 combining the time stamped video text signal and the delayed time stamped received AV signal based on the time stamps; and 
 outputting the combined time stamped video text signal and the time stamped received AV signal to a display. 
   
     
     
         8 . The system according to  claim 7 , wherein the step of generating text captions from the partitioned extracted audio signal over a first duration further comprises:
 comparing the generated text captions with a list of text obtained by a source of the AV signal to improve accuracy of the generated text captions.   
     
     
         9 . The system according to  claim 8 , wherein the list of text obtained by the source of the AV signal comprises text associated with the subject matter of the AV signal. 
     
     
         10 . The system according to  claim 7 , wherein the step of generating text captions from the partitioned extracted audio signal over a first duration further comprises:
 obtaining metadata from the AV signal;   generating a list of text that substantially matches the subject matter of the AV signal based on the obtained metadata;   comparing the generated text captions with the generated list of text based on the obtained metadata to improve accuracy of the generated text captions.   
     
     
         11 . The system according to  claim 7 , wherein the step of generating text captions from the partitioned extracted audio signal over a first duration further comprises:
 using artificial intelligence programming techniques to develop a list of text that substantially matches the subject matter of the AV signal based on the obtained metadata;   comparing the generated text captions with the AI developed list of text to improve accuracy of the generated text captions.   
     
     
         12 . The system according to  claim 11 , wherein the AI programming techniques comprises:
 Recurrent Neural Networks that are trained to suppress non-voice audio resulting in significantly improved voice signal-to-noise ratio (SNR) and clarity.   
     
     
         13 . The system according to  claim 7 , wherein the AV signal receiver, at least one processor and memory are part of an audio video display device.