Phrase-based dialogue modeling with particular application to creating a recognition grammar for a voice-controlled user interface

Abstract

The invention enables creation of grammar networks that can regulate, control, and define the content and scope of human-machine interaction in natural language voice user interfaces (NLVUI). The invention enables phrase-based modeling of generic structures of verbal interaction to be used for the purpose of automating part of the design of such grammar networks. Most particularly, the invention enables such grammar networks to be used in providing a voice-controlled user interface to human readable text data that is also machine-readable (such as a Web page, a word processing document, a PDF document, or a spreadsheet).

We claim:  
     
         1 . A method for creating a recognition grammar for use with an interactive user interface to human readable text data that is also machine readable, the interactive user interface being responsive to spoken input, the method comprising the steps of: 
 formulating an expression representing a part of the text data for each of one or more parts of the text data, wherein each formulated expression can be constructed as one or more combinations of one or more phrases in a phrase thesaurus; and    automatically using the phrase thesaurus to construct one or more equivalent expressions of one or more formulated expressions, wherein the recognition grammar comprises the collection of all of the expressions.    
     
     
         2 . A method as in  claim 1 , wherein the step of formulating an expression representing a part of the text data further comprises the step of formulating an expression representing an interactive part of the text data.  
     
     
         3 . A method as in  claim 1 , wherein the step of formulating an expression representing a part of the text data further comprises the step of formulating an expression representing a non-interactive part of the text data.  
     
     
         4 . A method as in  claim 1 , wherein the text data represents one or more Web pages.  
     
     
         5 . A method as in  claim 4 , wherein the step of formulating an expression further comprises the step of automatically parsing code representing the one or more Web pages to identify the one or more parts.  
     
     
         6 . A method as in  claim 5 , wherein one or more of the parts of the one or more Web pages comprise a system prompt indicating a type of interaction with an interactive part of a web page and a plurality of option values each representing a possible input to the interactive user interface for that type of interaction, the step of parsing further comprising the step of identifying the system prompt and the plurality of option values.  
     
     
         7 . A method as in  claim 6 , wherein the step of formulating an expression further comprises the steps of: 
 automatically identifying one or more phrases that correspond to a system prompt; and    automatically identifying, for each of a plurality of option values, one or more phrases that correspond to the option value.    
     
     
         8 . A method as in  claim 4 , wherein the code representing the one or more Web pages is expressed in a markup language.  
     
     
         9 . A method as in  claim 8 , wherein the code representing the one or more Web pages is expressed in HTML.  
     
     
         10 . A method as in  claim 8 , wherein the code epresenting the one or more Web pages is expressed in XML.  
     
     
         11 . A method as in  claim 4 , wherein the recognition grammar is expressed in VXML.  
     
     
         12 . A method as in  claim 1 , wherein the step of automatically using the phrase thesaurus to construct one or more equivalent expressions further comprises the steps of: 
 selecting a combination of one or more phrases representing the formulated expression, wherein the phrases of the selected combination of one or more phrases are original phrases of the formulated expression;    identifying an equivalent phrase for each of one or more original phrases of the formulated expression; and    producing a new combination of one or more phrases representing the formulated expression, the new combination including at least one of the identified equivalent phrases, wherein the new combination represents the equivalent expression.    
     
     
         13 . A method as in  claim 12 , wherein: 
 phrases in the phrase thesaurus have a probability of occurrence associated therewith;    one or more original phrases has a plurality of equivalent phrases; and    the step of identifying an equivalent phrase further comprises the step of selecting an equivalent phrase having the highest probability of occurrence.    
     
     
         14 . A method as in  claim 12 , wherein equivalent phrases are grouped in classes and each class of equivalent phrases has associated therewith a descriptor denoting a conceptual representation of the phrases contained in that phrase class, the method further comprising the step of tagging each equivalent expression with the descriptor or descriptors associated with phrases of the equivalent expression.  
     
     
         15 . A method as in  claim 1 , further comprising the step of translating the recognition grammar into a form that can be processed by a speech recognition system.  
     
     
         16 . A method as in  claim 1 , further comprising the step of manually editing the recognition grammar.  
     
     
         17 . A method as in  claim 1 , wherein expressions representing a plurality of parts of the text data are formulated and the phrase thesaurus is used to identify equivalent expressions for a plurality of formulated expressions.  
     
     
         18 . A system for creating a recognition grammar for use with an interactive user interface to human readable text data that is also machine readable, the interactive user interface being responsive to spoken input, the system comprising: 
 means for formulating an expression representing a part of the text data for each of one or more parts of the text data, wherein each formulated expression can be constructed as one or more combinations of one or more phrases in a phrase thesaurus; and    means for automatically using the phrase thesaurus to construct one or more equivalent expressions of one or more formulated expressions, wherein the recognition grammar comprises the collection of all of the expressions.    
     
     
         19 . A system as in  claim 18 , wherein the means for formulating an expression of each of one or more anticipated spoken inputs to the interface further comprises a graphical user interface device.  
     
     
         20 . A system as in  claim 18 , wherein the means for formulating an expression representing a part of the text data further comprises means for formulating an expression representing an interactive part of the text data.  
     
     
         21 . A system as in  claim 18 , wherein the means for formulating an expression representing a part of the text data further comprises means for formulating an expression representing a non-interactive part of the text data.  
     
     
         22 . A system as in  claim 18 , wherein the text data represents one or more Web pages.  
     
     
         23 . A system as in  claim 22 , wherein the means for formulating an expression further comprises means for automatically parsing code representing the one or more Web pages to identify the one or more parts.  
     
     
         24 . A system as in  claim 23 , wherein one or more of the parts of the one or more Web pages comprise a system prompt indicating a type of interaction with an interactive part of a web page and a plurality of option values each representing a possible input to the interactive user interface for that type of interaction, and wherein the means for parsing further comprises means for identifying the system prompt and the plurality of option values.  
     
     
         25 . A system as in  claim 24 , wherein the means for formulating an expression further comprises: 
 means for automatically identifying one or more phrases that correspond to a system prompt; and    means for automatically identifying, for each of a plurality of option values, one or more phrases that correspond to the option value.    
     
     
         26 . A system as in  claim 22 , wherein the means for automatically parsing further comprises means for automatically parsing code expressed in a markup language.  
     
     
         27 . A system as in  claim 26 , wherein the means for automatically parsing further comprises means for automatically parsing code expressed in HTML.  
     
     
         28 . A system as in  claim 26 , wherein the means for automatically parsing further comprises means for automatically parsing code expressed in XML.  
     
     
         29 . A system as in  claim 22 , wherein the recognition grammar is expressed in VXML.  
     
     
         30 . A system as in  claim 18 , further comprising data storage means for storing data representing the phrase thesaurus and the recognition grammar.  
     
     
         31 . A system as in  claim 30 , wherein the data storage means further stores data representing lexical items that can be used to complete a phrase template.  
     
     
         32 . A system as in  claim 30 , wherein the data storage means further stores data representing a probability of occurrence of phrases.  
     
     
         33 . A system as in  claim 32 , wherein the phrases are stored in the data storage means in accordance with the corresponding probability of occurrence.  
     
     
         34 . A system as in  claim 32 , wherein the means for automatically using the phrase thesaurus to construct equivalent phrases further comprises means for using the data representing a probability of occurrence of phrases to construct a probabilistic grammar.  
     
     
         35 . A system as in  claim 1 , wherein the means for automatically using the phrase thesaurus to construct equivalent phrases further comprises: 
 means for selecting a combination of one or more phrases representing a formulated expression, wherein the phrases of the selected combination of one or more phrases are original phrases of the formulated expression;    means for identifying an equivalent phrase for each of one or more original phrases of the formulated expression; and    means for producing a new combination of one or more phrases representing the formulated expression, the new combination including at least one of the identified equivalent phrases, wherein the new combination represents the equivalent expression.    
     
     
         36 . A system as in  claim 25 , wherein: 
 phrases in the phrase thesaurus have a probability of occurrence associated therewith;    one or more original phrases has a plurality of equivalent phrases; and    the means for identifying an equivalent phrase further comprises means for selecting an equivalent phrase having the highest probability of occurrence.    
     
     
         37 . A system as in  claim 35 , wherein equivalent phrases are grouped in classes and each class of equivalent phrases has associated therewith a descriptor denoting a conceptual representation of the phrases contained in that phrase class, the system further comprising means for tagging each equivalent expression with the descriptor or descriptors associated with phrases of the equivalent expression.  
     
     
         38 . A system as in  claim 18 , further comprising means for translating the recognition grammar into a form that can be processed by a speech recognition system.  
     
     
         39 . A system as in  claim 18 , further comprising means for manually editing the recognition grammar.  
     
     
         40 . A system as in  claim 18 , wherein expressions representing a plurality of parts of the text data are formulated and the phrase thesaurus is used to identify equivalent expressions for a plurality of formulated expressions.  
     
     
         41 . A computer readable storage medium encoded with one or more computer programs for creating a recognition grammar for use with an interactive user interface to human readable text data that is also machine readable, the interactive user interface being responsive to spoken input, the computer programs comprising: 
 instructions for formulating an expression representing a part of the text data for each of one or more parts of the text data, wherein each formulated expression can be constructed as one or more combinations of one or more phrases in a phrase thesaurus; and    instructions for automatically using the phrase thesaurus to construct one or more equivalent expressions of one or more formulated expressions, wherein the recognition grammar comprises the collection of all of the expressions.    
     
     
         42 . A computer readable storage medium as in  claim 41 , wherein the instructions for formulating an expression representing a part of the text data further comprise instructions for formulating an expression representing an interactive part of the text data.  
     
     
         43 . A computer readable storage medium as in  claim 41 , wherein the instructions for formulating an expression representing a part of the text data further comprise instructions for formulating an expression representing a non-interactive part of the text data.  
     
     
         44 . A computer readable storage medium as in  claim 41 , wherein the text data represents one or more Web pages.  
     
     
         45 . A computer readable storage medium as in  claim 44 , wherein the instructions for formulating an expression further comprise instructions for automatically parsing code representing the one or more Web pages to identify the one or more parts.  
     
     
         46 . A computer readable storage medium as in  claim 45 , wherein one or more of the parts of the one or more Web pages comprise a system prompt indicating a type of interaction with an interactive part of a web page and a plurality of option values each representing a possible input to the interactive user interface for that type of interaction, the instructions for parsing further comprising instructions for identifying the system prompt and the plurality of option values.  
     
     
         47 . A computer readable storage medium as in  claim 46 , wherein the instructions for formulating an expression further comprise: 
 instructions for automatically identifying one or more phrases that correspond to a system prompt; and    instructions for automatically identifying, for each of a plurality of option values, one or more phrases that correspond to the option value.    
     
     
         48 . A computer readable storage medium as in  claim 44 , wherein the instructions for automatically parsing further comprise instructions for automatically parsing code expressed in a markup language.  
     
     
         49 . A computer readable storage medium as in  claim 48 , wherein the instructions for automatically parsing further comprise instructions for automatically parsing code expressed in HTML.  
     
     
         50 . A computer readable storage medium as in  claim 48 , wherein the instructions for automatically parsing further comprise instructions for automatically parsing code expressed in XML.  
     
     
         51 . A computer readable storage medium as in  claim 44 , wherein the recognition grammar is expressed in VXML.  
     
     
         52 . A computer readable storage medium as in  claim 41 , wherein the instructions for automatically using the phrase thesaurus to construct equivalent phrases further comprise: 
 instructions for selecting a combination of one or more phrases representing a formulated expression, wherein the phrases of the selected combination of one or more phrases are original phrases of the formulated expression;    instructions for identifying an equivalent phrase for each of one or more original phrases of the formulated expression; and    instructions for producing a new combination of one or more phrases representing the formulated expression, the new combination including at least one of the identified equivalent phrases, wherein the new combination represents the equivalent expression.    
     
     
         53 . A computer readable storage medium as in  claim 52 , wherein: 
 phrases in the phrase thesaurus have a probability of occurrence associated therewith;    one or more original phrases has a plurality of equivalent phrases; and    the instructions for identifying an equivalent phrase further comprise instructions for selecting an equivalent phrase having the highest probability of occurrence.    
     
     
         54 . A computer readable storage medium as in  claim 52 , wherein equivalent phrases are grouped in classes and each class of equivalent phrases has associated therewith a descriptor denoting a conceptual representation of the phrases contained in that phrase class, the one or more computer programs further comprising instructions for tagging each equivalent expression with the descriptor or descriptors associated with phrases of the equivalent expression.  
     
     
         55 . A computer readable storage medium as in  claim 41 , further comprising instructions for translating the recognition grammar into a form that can be processed by a speech recognition system.  
     
     
         56 . A computer readable storage medium as in  claim 41 , further comprising instructions for manually editing the recognition grammar.  
     
     
         57 . A computer readable storage medium as in  claim 41 , wherein expressions representing a plurality of parts of the text data are formulated and the phrase thesaurus is used to identify equivalent expressions for a plurality of formulated expressions.