Augmentation and calibration of output from non-deterministic text generators by modeling its characteristics in specific environments

Abstract

Outputs of an automatic probabilistic event detection system, such as a fact extraction system, a speech-to-text engine or an automatic character recognition system, are matched with comparable results produced manually or by a different system. This comparison allows statistical modeling of the run-time behavior of the event detection system. This model can subsequently be used to give supplemental or replacement data for an output sequence of the system. In particular, the model can effectively calibrate the system for use with data of a particular statistical nature.

1 . A method of processing an original output of a text generation system, wherein the original output comprises a text transcript of speech after imperfect speech recognition, the method comprising the steps of:
 generating a statistical model of an observed output, wherein the observed output is produced by processing a training output of the text generation system and a reliable text transcript, wherein the training output comprises a text transcript of training speech after imperfect speech recognition and the reliable text transcript comprises a more accurate text transcription of the training speech than the training output; and   using the statistical model to process the original output and produce an alternate output, wherein the alternate output comprises a text transcript with at least one of supplementing and replacing at least part of the original output.   
   
   
       2 . A method as recited in  claim 1 , wherein data in the alternate output further comprises confidence assessments regarding parts of at least one of the original output and the alternate output, where the confidence assessments supplement data in the original output. 
   
   
       3 . A method as recited in  claim 1 , wherein data in the alternate output further comprises confidence assessments regarding parts of at least one of the original output and the alternate output, where the confidence assessments replace at least part of the original output. 
   
   
       4 . A method as recited in  claim 1 , wherein the alternate output further comprises information of a plurality of alternatives that can replace at least part of the original output. 
   
   
       5 . A method as recited in  claim 4 , wherein data in the alternate output further comprises confidence assessments regarding parts of the alternatives, where the confidence assessments supplement data in the original output. 
   
   
       6 . A method as recited in  claim 4 , wherein data in the alternate output further comprises confidence assessments regarding parts of the alternatives, where the confidence assessments replace at least part of the original output. 
   
   
       7 . A method as recited in  claim 1 , wherein said collecting comprises at least one of noting and estimating at least one detectable event that has transpired in correspondence with at least part of the original output. 
   
   
       8 . A method as recited in  claim 7 , wherein the detected events involve word recognition. 
   
   
       9 . A method as recited in  claim 1 , wherein the imperfect speech recognition operates on low-grade audio signals having word recognition precision below 50 percent. 
   
   
       10 . A method as recited in  claim 1 , wherein the reliable text transcript is produced by human transcription of the training speech. 
   
   
       11 . A method as recited in  claim 1 , wherein the alternate output further comprises at least one of
 an alternate recognition score for at least one of the words,   at least one alternate word that may have been one detectable event that transpired,   the at least one alternate word along with a recognition score for the at least one alternate word,   at least one alternate sequence of words that may have been another detectable event that transpired,   the at least one alternate sequence of words along with a recognition score for at least one word that is part of the at least one alternate sequence of words,   an indication that no detectable event has transpired,   a word lattice describing a plurality of alternatives for detectable word sequences, and   the word lattice along with a recognition score for at least one among
 at least one word in the detectable word sequences, 
 at least one path in the word lattice, and 
 at least one edge in the word lattice. 
   
   
   
       12 . A method as recited in  claim 1 , wherein said using comprises:
 building a first model modeling behavior as a process with at least one inner state, and inferring the at least one inner state of the process from the observed outputs;   building a second model, based on statistics obtained by said generating, to infer data to at least one of supplement and replace at least part of the original output from the at least one inner state of the process in the first model;   combining the first and second models to form a function for converting the original output into the alternate output; and   using the function on the original output to create the alternate output.   
   
   
       13 . A method as recited in  claim 12 , wherein the process in said first model is one of a Generalized Hidden Markov process and a special case of a Generalized Hidden Markov process. 
   
   
       14 . A method as recited in  claim 12 ,
 wherein the second model is a parametric model, and   wherein said building of the second model uses at least one direct parametric estimation technique for inferring from at least one of the inner states.   
   
   
       15 . A method as recited in  claim 12 , wherein at least one of said building and combining uses Bayesian methods. 
   
   
       16 . A method as recited in  claim 1 , further comprising repeating said collecting on several statistically different training materials. 
   
   
       17 . A system for processing an original output of a text generation system, wherein the original output comprises a text transcript of speech after imperfect speech recognition, the system comprising:
 a statistical modeling component for generating a statistical model of an observed output, wherein the observed output is produced by processing a training output of the text generation system and a reliable text transcript, wherein the training output comprises a text transcript of training speech after imperfect speech recognition and the reliable text transcript comprises a more accurate text transcription of the training speech than the training output; and   processing means for using the statistical model to process the original output and produce an alternate output, wherein the alternate output comprises a text transcript with at least one of supplementing and replacing at least part of the original output.   
   
   
       18 . The system as recited in  claim 1 , wherein data in the alternate output further comprises confidence assessments regarding parts of at least one of the original output and the alternate output, where the confidence assessments supplement data in the original output. 
   
   
       19 . The system as recited in  claim 1 , wherein data in the alternate output further comprises confidence assessments regarding parts of at least one of the original output and the alternate output, where the confidence assessments replaces data in the original output. 
   
   
       20 . A computer readable medium storing instructions for controlling at least one computer system to perform a method of processing an original output of a text generation system, wherein the original output comprises a text transcript of speech after imperfect speech recognition, comprising:
 generating a statistical model of an observed output, wherein the observed output is produced by processing a reliable text transcript and a training output of the text generation system, wherein the training output comprises a text transcript of training speech after imperfect speech recognition and the reliable text transcript comprises a more accurate text transcript of the training speech than the training output; and   using the statistical model to process the original output and produce an alternate output, wherein the alternate output comprises a text transcript with at least one of supplementing and replacing at least part of the original output.