Apparatus and method for integrated phrase-based and free-form speech-to-speech translation

Abstract

An apparatus and method that integrates both phrase-based and free-form speech-to-speech translation approaches using probability models. The starting step of the method is to receive vocal communication in a source language. Then store the received vocal communication. Then decipher the content of the vocal communication. Then locate in a multilingual dictionary module the corresponding translation of the deciphered vocal communication provided a preset sentence exists in a speech recognition module for the vocal communication. Then translate the vocal communication into the target language provided there is no corresponding translation located in the multilingual dictionary module. Then synthesize the translated target language when there is no corresponding translation for the vocal communication in the multilingual dictionary module. Then store the sound of the translated target language. Then play the sound of the translated target language.

1 . A translation apparatus that integrates both phrase-based and free-form speech-to-speech translation approaches using probability models, comprising:
 a housing;   a first communication member embedded in the housing, the first communication member configured to receive vocal communication from a user speaking in a source language;   an input audio buffer operably connected to the first communication member, the input audio buffer configured to store the vocal communication transferred from the first communication member;   a speech recognition module configured to recognize a plurality of preset sentences and a plurality of free-form sentences, the speech recognition module further configured to receive the vocal communication from the input audio buffer and decipher the content of the vocal communication;   a multilingual dictionary module operably connected to the speech recognition module, the multilingual dictionary module configured to locate a corresponding translation of the vocal communication provided a preset sentence exists in the speech recognition module for the vocal communication;   a translation module operably connected to the speech recognition module, the translation module configured to translate the vocal communication into the target language provided there is no corresponding translation located in the multilingual dictionary module;   a synthesis module operably connected to the translation module, the synthesis module configured to synthesize text to speech (TTS) for the translated target language;   an output audio buffer operably connected to the multilingual dictionary module and the synthesis module, the output audio buffer configured to store prerecorded speech sounds from the multilingual dictionary module corresponding to the translated vocal communication and further configured to store the synthesized speech sounds from the synthesis module corresponding to the translated target language; and   a second communication member embedded in the housing and operably connected to the output audio buffer, the second communication member configured to receive and play the stored sounds in the output audio buffer.   
     
     
         2 . The apparatus of  claim 1 , wherein the TTS synthesis occurs for at least one of, (i) free-form TTS for handling text and (ii) phrase-based large inventory pre-recorded sentences and phrases. 
     
     
         3 . The apparatus of  claim 2 , further including a selection switch embedded in the housing for allowing the user to selectively choose between either one of the phrase-based approach and the free-form approach of speech-to-speech translation. 
     
     
         4 . The apparatus of  claim 3 , wherein the speech recognition module configured to decipher the form of the vocal communication yields the best recognition output when ŵ=arg w max p(X|W, λ w ), where λ w  is the probability weight for a given text W and 1≧λ w ≧0. 
     
     
         5 . The apparatus of  claim 4 , wherein the combination weight between the phrase-based approach and the free-form approach of speech-to-speech translation may be either one of, (i) user adjusted and (ii) automatically optimized due to previously recorded conversations. 
     
     
         6 . The apparatus of  claim 5 , further including an on/off switch embedded in the housing. 
     
     
         7 . A method that integrates both phrase-based and free-form speech-to-speech translation approaches using probability models, comprising:
 receiving vocal communication in a source language;   storing the received vocal communication;   deciphering the content of the vocal communication;   locating in a multilingual dictionary module the corresponding translation of the deciphered vocal communication provided a preset sentence exists in a speech recognition module for the vocal communication;   translating the vocal communication into the target language provided there is no corresponding translation located in the multilingual dictionary module;   synthesizing the translated target language when there is no corresponding translation for the vocal communication in the multilingual dictionary module;   storing the sound of the translated target language; and   playing the sound of the translated target language.