System and method for translating real-time speech using segmentation based on conjunction locations

Abstract

A system, method and computer-readable storage device which balance latency and accuracy of machine translations by segmenting the speech upon locating a conjunction. The system, upon receiving speech, will buffer speech until a conjunction is detected. Upon detecting a conjunction, the speech received until that point is segmented. The system then continues performing speech recognition on the segment, searching for the next conjunction, while simultaneously initiating translation of the segment. Upon translating the segment, the system converts the translation to a speech output, allowing a user to hear an audible translation of the speech originally heard.

We claim: 
     
         1 . A method comprising:
 receiving speech in a first language, wherein the speech has a first portion, a second portion, and a conjunction which separates the first portion from the second portion;   performing, via a processor, a speech recognition process on the first portion of the speech until the conjunction is recognized by the speech recognition process; and   when an accuracy of translating the first portion of the speech to a second language is below a threshold, increasing segment lengths by reducing conjunctions searched for, increasing a maximum number of words per segment, and identifying a second conjunction which defines a new first portion of the speech and a new second portion of the speech.   
     
     
         2 . The method of  claim 1 , wherein upon identifying the second conjunction, the method further comprises:
 segmenting the speech by generating a new speech segment, the new speech segment comprising the new first portion of the speech to the second conjunction;   performing a translation of the new speech segment from the first language to the second language, to yield a new translated speech segment;   receiving the new second portion of the speech; and   outputting the new translated speech segment.   
     
     
         3 . The method of  claim 1 , wherein the conjunction comprises one of “and” and “or”. 
     
     
         4 . The method of  claim 3 , wherein the speech recognition process comprises identifying punctuation using a punctuation classifier. 
     
     
         5 . The method of  claim 2 , further comprising repeating the segmenting, the performing and the outputting with respect to the second portion of the speech. 
     
     
         6 . The method of  claim 1 , wherein the processor searches for fewer conjunctions when an accuracy is above a threshold. 
     
     
         7 . The method of  claim 2 , further comprising outputting additional translated speech immediately after the outputting of the new translated speech segment such that an output break between the new translated speech segment and the additional translated speech matches a pause in the speech corresponding to an input break between the first portion and the second portion of the speech. 
     
     
         8 . A system comprising:
 a processor; and   a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
 receiving speech in a first language, wherein the speech has a first portion, a second portion, and a conjunction which separates the first portion from the second portion; 
 performing a speech recognition process on the first portion of the speech until the conjunction is recognized by the speech recognition process; and 
 when an accuracy of translating the first portion of the speech to a second language is below a threshold, increasing segment lengths by reducing conjunctions searched for, increasing a maximum number of words per segment, and identifying a second conjunction which defines a new first portion of the speech and a new second portion of the speech. 
   
     
     
         9 . The system of  claim 8 , wherein the computer-readable storage medium stores instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
 upon identifying the second conjunction:   segmenting the speech by generating a new speech segment, the new speech segment comprising the new first portion of the speech to the second conjunction;   performing a translation of the new speech segment from the first language to the second language, to yield a new translated speech segment;   receiving the new second portion of the speech; and   outputting the new translated speech segment.   
     
     
         10 . The system of  claim 8 , wherein the conjunction comprises one of “and” and “or”. 
     
     
         11 . The system of  claim 10 , wherein the speech recognition process comprises identifying punctuation using a punctuation classifier. 
     
     
         12 . The system of  claim 9 , wherein the computer-readable storage medium stores instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
 repeating the segmenting, the performing and the outputting with respect to the second portion of the speech.   
     
     
         13 . The system of  claim 8 , wherein the processor searches for fewer conjunctions when an accuracy is above a threshold. 
     
     
         14 . The system of  claim 9 , wherein the computer-readable storage medium stores instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
 outputting additional translated speech immediately after the outputting of the new translated speech segment such that an output break between the new translated speech segment and the additional translated speech matches a pause in the speech corresponding to an input break between the first portion and the second portion of the speech.   
     
     
         15 . A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
 receiving speech in a first language, wherein the speech has a first portion, a second portion, and a conjunction which separates the first portion from the second portion;   performing a speech recognition process on the first portion of the speech until the conjunction is recognized by the speech recognition process; and   when an accuracy of translating the first portion of the speech to a second language is below a threshold, increasing segment lengths by reducing conjunctions searched for, increasing a maximum number of words per segment, and identifying a second conjunction which defines a new first portion of the speech and a new second portion of the speech.   
     
     
         16 . The computer-readable storage device of  claim 15 , wherein the computer-readable storage device stores additional instructions which, when executed by the computing device, cause the computing device to perform operations further comprising:
 upon identifying the second conjunction:
 segmenting the speech by generating a new speech segment, the new speech segment comprising the new first portion of the speech to the second conjunction; 
 performing a translation of the new speech segment from the first language to the second language, to yield a new translated speech segment; 
 receiving the new second portion of the speech; and 
 outputting the new translated speech segment. 
   
     
     
         17 . The computer-readable storage device of  claim 15 , wherein the conjunction comprises one of “and” and “or”. 
     
     
         18 . The computer-readable storage device of  claim 17 , wherein the speech recognition process comprises identifying punctuation using a punctuation classifier. 
     
     
         19 . The computer-readable storage device of  claim 16 , wherein the computer-readable storage device stores additional instructions which, when executed by the computing device, cause the computing device to perform operations further comprising:
 repeating the segmenting, the performing and the outputting with respect to the second portion of the speech.   
     
     
         20 . The computer-readable storage device of  claim 16 , wherein the computer-readable storage device stores additional instructions which, when executed by the computing device, cause the computing device to perform operations further comprising:
 outputting additional translated speech immediately after the outputting of the new translated speech segment such that an output break between the new translated speech segment and the additional translated speech matches a pause in the speech corresponding to an input break between the first portion and the second portion of the speech.