Text-to-speech (tts) processing

Abstract

A speech model is trained using multi-task learning. A first task may correspond to how well predicted audio matches training audio; a second task may correspond to a metric of perceived audio quality. The speech model may include, during training, layers related to the second task that are discarded at runtime.

1 .- 20 . (canceled) 
     
     
         21 . A computer-implemented method, comprising:
 receiving metadata corresponding to a desired synthetic voice, the metadata including audio data and input data representing text;   processing the input data to determine phonetic data;   processing the audio data to determine acoustic feature data; and   using the phonetic data and the acoustic feature data to configure a trained speech model configured to output audio data corresponding to the desired synthetic voice, wherein the trained speech model is configured using:
 a first task, and 
 a second task corresponding to increasing a metric of perceived quality of audio output data. 
   
     
     
         22 . The computer-implemented method of  claim 21 , wherein the trained speech model is configured to input meaning representation data and output audio output data. 
     
     
         23 . The computer-implemented method of  claim 21 , further comprising:
 configuring a first plurality of values using a first section of a first model in accordance with the first task;   configuring a second plurality of values using a second section of the first model in accordance with the second task; and   configuring the trained speech model to include the first section of the first model but not the second section of the first model.   
     
     
         24 . The computer-implemented method of  claim 23 , further comprising:
 configuring the trained speech model to include a hidden layer comprising the first plurality of values and the second plurality of values.   
     
     
         25 . The computer-implemented method of  claim 24 , further comprising, during increasing the metric of perceived quality, reducing a number of nodes of the hidden layer. 
     
     
         26 . The computer-implemented method of  claim 25 , further comprising determining a second speech model using an output of the hidden layer. 
     
     
         27 . The computer-implemented method of  claim 23 , wherein the first section of the first model corresponds to a first section of a conditioning model, and wherein the second section of the first model corresponds to a second section of the conditioning model. 
     
     
         28 . The computer-implemented method of  claim 23 , wherein:
 the first section of the first model corresponds to a first section of an output model; and   the second section of the first model corresponds to a second section of the output model.   
     
     
         29 . The computer-implemented method of  claim 21 , wherein the first task includes reducing a difference between the audio output data and corresponding training data. 
     
     
         30 . The computer-implemented method of  claim 21 , wherein increasing the metric of perceived quality comprises computing, using audio output data, a perceptual evaluation of speech quality (PESQ) standard. 
     
     
         31 . The computer-implemented method of  claim 21 , wherein increasing the metric of perceived quality comprises computing, using the audio output data, mel-frequency cepstrum (MFC) data. 
     
     
         32 . A system comprising:
 at least one processor; and   at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:
 receive metadata corresponding to a desired synthetic voice, the metadata including audio data and input data representing text; 
 process the input data to determine phonetic data; 
 process the audio data to determine acoustic feature data; and 
 use the phonetic data and the acoustic feature data to configure a trained speech model configured to output audio data corresponding to the desired synthetic voice, wherein the trained speech model is configured using:
 a first task, and 
 a second task corresponding to increasing a metric of perceived quality of audio output data. 
 
   
     
     
         33 . The system of  claim 32 , wherein the trained speech model is configured to input meaning representation data and output audio output data. 
     
     
         34 . The system of  claim 32 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 configure a first plurality of values using a first section of a first model in accordance with the first task;   configure a second plurality of values using a second section of the first model in accordance with the second task; and   configure the trained speech model to include the first section of the first model but not the second section of the first model.   
     
     
         35 . The system of  claim 34 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 configure the trained speech model to include a hidden layer comprising the first plurality of values and the second plurality of values.   
     
     
         36 . The system of  claim 35 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to, during increasing the metric of perceived quality, reduce a number of nodes of the hidden layer. 
     
     
         37 . The system of  claim 36 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to determine a second speech model using an output of the hidden layer. 
     
     
         38 . The system of  claim 34 , wherein the first section of the first model corresponds to a first section of a conditioning model, and wherein the second section of the first model corresponds to a second section of the conditioning model. 
     
     
         39 . The system of  claim 34 , wherein:
 the first section of the first model corresponds to a first section of an output model; and   the second section of the first model corresponds to a second section of the output model.   
     
     
         40 . The system of  claim 34 , wherein the first task includes reducing a difference between the audio output data and corresponding training data.