System and Method for Optimizing Speech Recognition and Natural Language Parameters with User Feedback

Abstract

Disclosed herein are systems, methods, and non-transitory computer-readable storage media for assigning saliency weights to words of an ASR model. The saliency values assigned to words within an ASR model are based on human perception judgments of previous transcripts. These saliency values are applied as weights to modify an ASR model such that the results of the weighted ASR model in converting a spoken document to a transcript provide a more accurate and useful transcription to the user.

We claim: 
     
         1 . A method comprising:
 converting, via one or more processors, speech to text using a first automatic speech recognition model, to yield a first transcript;   converting, via the one or more processors, the speech to text using a second automatic speech recognition model, to yield a second transcript;   receiving, from a user, an indication of an accuracy of at least one of the first transcript or the second transcript; and   updating at least one of the first automatic speech recognition model or the second automatic speech recognition model based on the indication of the accuracy received from the user.   
     
     
         2 . The method of  claim 1 , further comprising:
 weighting the first automatic speech recognition model, to yield a weighted first automatic speech recognition model;   weighting the second automatic speech recognition model, to yield a weighted second automatic speech recognition model;   converting, via a processor, speech to text using the weighted first automatic speech recognition model, to yield the first transcript; and   converting, via the processor, the speech to text using the weighted second automatic speech recognition model, to yield the second transcript, wherein the indication comprises a judgment of perceived accuracy of one of the first transcript and the second transcript.   
     
     
         3 . The method of  claim 2 , further comprising:
 updating, via the processor, one of the weighted first automatic speech recognition model and the weighted second automatic speech recognition model based on the judgment.   
     
     
         4 . The method of  claim 2 , wherein the weighting of the first automatic speech recognition model and the weighting of the second automatic speech recognition model is based on a context of the speech. 
     
     
         5 . The method of  claim 2 , wherein the judgment is received for the first transcript of the second transcript with a highest score. 
     
     
         6 . The method of  claim 2 , wherein the weighting of the first automatic speech recognition model and the weighting of the second automatic speech recognition model is based on a user profile. 
     
     
         7 . The method of  claim 6 , wherein the user profile comprises a list of contexts. 
     
     
         8 . The method of  claim 6 , wherein the user profile comprises a previous communication history. 
     
     
         9 . The method of  claim 2 , wherein the weighted first automatic speech recognition model and the weighted second automatic speech recognition model each contain saliency weights to words in the speech. 
     
     
         10 . The method of  claim 9 , wherein a high saliency weight indicates a high predicted importance to the user. 
     
     
         11 . The method of  claim 10 , wherein the processor spends additional effort converting high saliency text. 
     
     
         12 . A system comprising:
 a processor; and   a computer-readable storage device having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
 converting speech to text using a first automatic speech recognition model, to yield a first transcript; 
 converting the speech to text using a second automatic speech recognition model, to yield a second transcript; 
 receiving, from a user, an indication of an accuracy of at least one of the first transcript or the second transcript; and 
 updating at least one of the first automatic speech recognition model or the second automatic speech recognition model based on the indication of the accuracy received from the user. 
   
     
     
         13 . The system of  claim 12 , wherein the computer-readable storage device stores additional instructions stored which, when executed by the processor, cause the processor to perform operations further comprising:
 weighting the first automatic speech recognition model, to yield a weighted first automatic speech recognition model;   weighting the second automatic speech recognition model, to yield a weighted second automatic speech recognition model;   converting, via a processor, speech to text using the weighted first automatic speech recognition model, to yield the first transcript; and   converting, via the processor, the speech to text using the weighted second automatic speech recognition model, to yield the second transcript, wherein the indication comprises a judgment of perceived accuracy of one of the first transcript and the second transcript.   
     
     
         14 . The system of  claim 13 , wherein the computer-readable storage device stores additional instructions stored which, when executed by the processor, cause the processor to perform operations further comprising:
 updating, via the processor, one of the weighted first automatic speech recognition model and the weighted second automatic speech recognition model based on the judgment.   
     
     
         15 . The system of  claim 13 , wherein the weighting of the first automatic speech recognition model and the weighting of the second automatic speech recognition model is based on a context of the speech. 
     
     
         16 . The system of  claim 13 , wherein the judgment is received for the first transcript of the second transcript with a highest score. 
     
     
         17 . The system of  claim 13 , wherein the weighting of the first automatic speech recognition model and the weighting of the second automatic speech recognition model is based on a user profile. 
     
     
         18 . The system of  claim 17 , wherein the user profile comprises a list of contexts. 
     
     
         19 . The system of  claim 18 , wherein the user profile comprises a previous communication history. 
     
     
         20 . The system of  claim 13 , wherein the weighted first automatic speech recognition model and the weighted second automatic speech recognition model each contain saliency weights to words in the speech.