Neural speech-to-meaning

Abstract

A neural speech-to-meaning system is trained on speech audio expressing specific intents. The system receives speech audio and produces indications of when the speech in the audio matches the intent. Intents may include variables that can have a large range of values, such as the names of places. The neural speech-to-meaning system simultaneously recognizes enumerated values of variables and general intents. Recognized variable values can serve as arguments to API requests made in response to recognized intents. Accordingly, neural speech-to-meaning supports voice virtual assistants that serve users based on API hits.

What is claimed is: 
     
         1 . A machine for recognizing an intent in speech audio, the machine comprising:
 a variable recognizer that processes speech audio features, computes a probability of the speech audio having any of a plurality of enumerated variable values, and outputs the value with the highest probability; and   an intent recognizer that processes speech audio features, computes a probability of the speech audio having the intent, and in response to the probability being above an intent threshold, produces a request for a virtual assistant action.   
     
     
         2 . The machine of  claim 1  wherein:
 the variable recognizer indicates the probability of the speech audio having an enumerated variable value; and 
 the intent recognizer conditions its output of a request for an action on the probability of the speech audio having an enumerated variable values. 
 
     
     
         3 . The machine of  claim 2  wherein the conditioning is based on a delayed indication of the probability of the speech audio having an enumerated variable value. 
     
     
         4 . The machine of  claim 1  wherein the intent recognizer conditions its output of a request for an action on which value of the variable has the highest probability. 
     
     
         5 . The machine of  claim 4  wherein the conditioning is based on a delayed indication of which value of the variable has the highest probability. 
     
     
         6 . The machine of  claim 1  wherein one of the recognizers produces a score and the other recognizer is called in response to the score being above a score threshold. 
     
     
         7 . The machine of  claim 1  further comprising a domain recognizer that processes speech audio features and computes a probability of the speech audio referring to a specific domain, wherein the intent recognizer is associated with the domain and called in response to the probability of the speech audio referring to a specific domain being above a domain threshold. 
     
     
         8 . The machine of  claim 1  wherein no human-readable speech transcription is computed. 
     
     
         9 . The machine of  claim 1  further comprising a network client with access to a web API, wherein, in response to the intent recognizer producing a request for a virtual assistant action, the network client performs a request to the web API, the request having, as an argument, the value output by the variable recognizer. 
     
     
         10 . The machine of  claim 9  further comprising a speech synthesis engine, wherein, in response to receiving a response from the web API, the speech synthesis engine synthesizes speech audio containing information from the web API response and outputs the synthesized speech audio for a user of the virtual assistant. 
     
     
         11 . A method of recognizing an intent from speech audio by a computer system, the method comprising:
 obtaining speech audio;   processing features of the speech audio to compute a probability of the speech audio having any of a plurality of enumerated variable values;   outputting the value with the highest probability;   processing the features of the speech audio to compute a probability of the speech audio having the intent; and   in response to the probability being above an intent threshold, outputting a request for a virtual assistant action.   
     
     
         12 . The method of  claim 11  wherein outputting a request is conditioned on the probability of the variable with the highest probability. 
     
     
         13 . The method of  claim 12  wherein the probability is delayed. 
     
     
         14 . The method of  claim 11  wherein outputting a request is conditioned on which value of the variable has the highest probability. 
     
     
         15 . The method of  claim 14  wherein an indication of the value of the variable is delayed. 
     
     
         16 . The method of  claim 11  wherein one of the probability computations is performed in response to the other probability computation having a result that is above a threshold. 
     
     
         17 . The method of  claim 11  further comprising processing features of the speech audio to compute a probability of the speech audio referring to a specific domain, wherein the intent is associated with the domain and computing the probability of the speech audio having the intent is performed in response to the probability of the speech audio referring to the domain being above a threshold. 
     
     
         18 . The method of  claim 11  wherein no human-readable speech transcription is computed. 
     
     
         19 . The method of  claim 11  wherein outputting a request for a virtual assistant action is performed by making a request to a web API, the request having as an argument, the value with the highest probability. 
     
     
         20 . The method of  claim 19  further comprising receiving a response from the web API; and synthesizing speech audio containing information from the web API response.