Device and method for clarifying dysarthria voices

Abstract

A device and a method for clarifying dysarthria voices is disclosed. Firstly, a dysarthria voice signal is received and framed to generate dysarthria frames. Then, the dysarthria frames are received to retrieve dysarthria features. Finally, the dysarthria features are received. Without receiving phases corresponding to the dysarthria features, the dysarthria features are converted into an intelligent voice signal based on an intelligent voice conversion model. The intelligent voice conversion model is not trained by the dynamic time warping (DTW). The present invention avoids the phase distortion of the voice signal and provides more natural and clarified voices with low noise.

What is claimed is: 
     
       1. A device for clarifying dysarthria voices comprising:
 one or more first processors configured to:
 receive and frame a dysarthria voice signal to generate dysarthria frames; 
 extract dysarthria features from the dysarthria frames; and 
 convert the dysarthria features into an intelligent voice signal based on an intelligent voice conversion model without receiving phases corresponding to the dysarthria features; 
 
 wherein the intelligent voice conversion model is not trained based on dynamic time warping (DTW); 
 wherein the intelligent voice conversion model is trained by an intelligent voice training system, and the intelligent voice training system comprises:
 one or more second processors configured to:
 receive and frame a dysarthria corpus corresponding to the dysarthria voice signal to generate dysarthria corpus frames; 
 extract from the dysarthria corpus frames dysarthria corpus features corresponding to the dysarthria features; 
 convert the dysarthria corpus features into reference corpus features corresponding to the intelligent voice signal based on a feature mapping model; 
 convert the reference corpus features into reference corpus frames based on a voice synthesizing model; and 
 train the intelligent voice conversion model based on the reference corpus frames and the dysarthria corpus features. 
 
 
 
     
     
       2. The device for clarifying dysarthria voices according to  claim 1 , wherein the feature mapping model is trained by a feature mapping training system, and the feature mapping training system comprises:
 one or more third processors configured to:
 receive, frame, and align the dysarthria corpus and a reference corpus to generate the dysarthria corpus frames and the reference corpus frames, wherein the dysarthria corpus frames and the reference corpus frames are aligned to each other, and the reference corpus corresponds to the intelligent voice signal; 
 respectively extract the dysarthria corpus features and the reference corpus features from the dysarthria corpus frames and the reference corpus frames; and 
 train the feature mapping model based on the dysarthria corpus features and the reference corpus features. 
 
 
     
     
       3. The device for clarifying dysarthria voices according to  claim 1 , wherein the voice synthesizing model is trained by a voice synthesizing training system, and the voice synthesizing training system comprises:
 one or more third processors configured to:
 receive and frame a reference corpus to generate the reference corpus frames, wherein the reference corpus corresponds to the intelligent voice signal; 
 extract the reference corpus features from the reference corpus frames; and 
 train the voice synthesizing model based on the reference corpus frames and the reference corpus features. 
 
 
     
     
       4. The device for clarifying dysarthria voices according to  claim 1 , wherein the intelligent voice conversion model comprises a feature mapping model and a voice synthesizing model, and the first processor is configured to:
 convert the dysarthria features into reference features based on the feature mapping model; and 
 convert the reference features into the intelligent voice signal based on the voice synthesizing model. 
 
     
     
       5. The device for clarifying dysarthria voices according to  claim 4 , wherein the feature mapping model is trained by a feature mapping training system, and the feature mapping training system comprises:
 one or more third processors configured to:
 receive, frame, and align a dysarthria corpus and a reference corpus to generate dysarthria corpus frames and reference corpus frames that are aligned to each other, wherein the dysarthria corpus corresponds to the dysarthria voice signal, and the reference corpus corresponds to the intelligent voice signal; 
 respectively extract dysarthria corpus features and reference corpus features from the dysarthria corpus frames and the reference corpus frames, wherein the dysarthria corpus features and the reference corpus features respectively correspond to the dysarthria features and the reference features; and 
 train the feature mapping model based on the dysarthria corpus features and the reference corpus features. 
 
 
     
     
       6. The device for clarifying dysarthria voices according to  claim 4 , wherein the voice synthesizing model is trained by a voice synthesizing training system, and the voice synthesizing training system comprises:
 one or more third processors configured to:
 receive and frame a reference corpus to generate reference corpus frames, wherein the reference corpus corresponds to the intelligent voice signal; 
 extract reference corpus features corresponding to the reference features from the reference corpus frames; and 
 train the voice synthesizing model based on the reference corpus frames and the reference corpus features. 
 
 
     
     
       7. The device for clarifying dysarthria voices according to  claim 1 , wherein the dysarthria features comprise at least one of a log power spectrum (LPS), a Mel spectrum, a fundamental frequency, a Mel-frequency cepstral coefficient, and an aperiodicity, and the intelligent voice conversion model comprises a WaveNet or a Wave recurrent neural network (RNN). 
     
     
       8. The device for clarifying dysarthria voices according to  claim 1 , wherein the dysarthria features comprise log power spectrums, and the intelligent voice converter is configured to convert the dysarthria features into the intelligent voice signal using an inverse Fourier transform. 
     
     
       9. The device for clarifying dysarthria voices according to  claim 1 , wherein the dysarthria features comprise a log power spectrum (LPS), a Mel spectrum, a fundamental frequency, a Mel-frequency cepstral coefficient, and an aperiodicity, and the intelligent voice converter is a vocoder. 
     
     
       10. A method for clarifying dysarthria voices comprising:
 receiving and framing a dysarthria voice signal to generate dysarthria frames; 
 receiving the dysarthria frames and extracting dysarthria features from the dysarthria frames; and 
 receiving the dysarthria features and converting the dysarthria features into an intelligent voice signal based on an intelligent voice conversion model without receiving phases corresponding to the dysarthria features; 
 wherein the intelligent voice conversion model is not trained based on dynamic time warping (DTW); 
 wherein a method for training the intelligent voice conversion model comprises:
 receiving and framing a dysarthria corpus corresponding to the dysarthria voice signal to generate dysarthria corpus frames; 
 receiving the dysarthria corpus frames and extract from the dysarthria corpus frames dysarthria corpus features corresponding to the dysarthria features; 
 receiving the dysarthria corpus features and converting the dysarthria corpus features into reference corpus features corresponding to the intelligent voice signal based on a feature mapping model; 
 receiving the reference corpus features and converting the reference corpus features into reference corpus frames based on a voice synthesizing model; and 
 receiving the reference corpus frames and the dysarthria corpus features and training the intelligent voice conversion model based on the reference corpus frames and the dysarthria corpus features. 
 
 
     
     
       11. The method for clarifying dysarthria voices according to  claim 10 , wherein a method for training the feature mapping model comprises:
 receiving, framing, and aligning the dysarthria corpus and a reference corpus to generate the dysarthria corpus frames and the reference corpus frames, wherein the dysarthria corpus frames and the reference corpus frames are aligned to each other, and the reference corpus corresponds to the intelligent voice signal; 
 receiving the dysarthria corpus frames and the reference corpus frames and respectively extracting the dysarthria corpus features and the reference corpus features from the dysarthria corpus frames and the reference corpus frames; and 
 receiving the dysarthria corpus features and the reference corpus features and training the feature mapping model based on the dysarthria corpus features and the reference corpus features. 
 
     
     
       12. The method for clarifying dysarthria voices according to  claim 10 , wherein a method for training the voice synthesizing model comprises:
 receiving and framing a reference corpus to generate the reference corpus frames, wherein the reference corpus corresponds to the intelligent voice signal; 
 receiving the reference corpus frames and extracting the reference corpus features from the reference corpus frames; and 
 receiving the reference corpus frames and the reference corpus features and training the voice synthesizing model based on the reference corpus frames and the reference corpus features. 
 
     
     
       13. The method for clarifying dysarthria voices according to  claim 10 , wherein the intelligent voice conversion model comprises a feature mapping model and a voice synthesizing model, and the step of receiving the dysarthria features and converting the dysarthria features into the intelligent voice signal based on the intelligent voice conversion model without receiving the phases comprises:
 receiving the dysarthria features and converting the dysarthria features into reference features based on the feature mapping model; and 
 receiving the reference features and convert the reference features into the intelligent voice signal based on the voice synthesizing model. 
 
     
     
       14. The method for clarifying dysarthria voices according to  claim 13 , wherein a method for training the feature mapping model comprises:
 receiving, framing, and aligning a dysarthria corpus and a reference corpus to generate dysarthria corpus frames and reference corpus frames that are aligned to each other, wherein the dysarthria corpus corresponds to the dysarthria voice signal, and the reference corpus corresponds to the intelligent voice signal; 
 receiving the dysarthria corpus frames and the reference corpus frames and respectively extracting dysarthria corpus features and reference corpus features from the dysarthria corpus frames and the reference corpus frames, wherein the dysarthria corpus features and the reference corpus features respectively correspond to the dysarthria features and the reference features; and 
 receiving the dysarthria corpus features and the reference corpus features and training the feature mapping model based on the dysarthria corpus features and the reference corpus features. 
 
     
     
       15. The method for clarifying dysarthria voices according to  claim 13 , wherein a method for training the voice synthesizing model comprises:
 receiving and framing a reference corpus to generate reference corpus frames, wherein the reference corpus corresponds to the intelligent voice signal; 
 receiving the reference corpus frames and extracting reference corpus features corresponding to the reference corpus from the reference corpus frames; and 
 receiving the reference corpus frames and the reference corpus features and training the voice synthesizing model based on the reference corpus frames and the reference corpus features. 
 
     
     
       16. The method for clarifying dysarthria voices according to  claim 10 , wherein the dysarthria features comprise at least one of a log power spectrum (LPS), a Mel spectrum, a fundamental frequency, a Mel-frequency cepstral coefficient, and an aperiodicity, and the intelligent voice conversion model comprises a WaveNet or a Wave recurrent neural network (RNN).