Semantic parsing using deep neural networks for predicting canonical forms

Abstract

A method for predicting a canonical form for an input text sequence includes predicting the canonical form with a neural network model. The model includes an encoder, which generates a first representation of the input text sequence based on a representation of n-grams in the text sequence and a second representation of the input text sequence generated by a first neural network. The model also includes a decoder which sequentially predicts terms of the canonical form based on the first and second representations and a predicted prefix of the canonical form. The canonical form can be used, for example, to query a knowledge base or to generate a next utterance in a discourse.

What is claimed is: 
     
       1. A method comprising:
 providing a neural network model which has been trained to predict a canonical form, containing a sequence of words, for an input text sequence, containing a sequence of words, the neural network model comprising:
 an encoder which generates a first representation of the input text sequence based on a representation of n-grams in the text sequence, the encoder including a first neural network which reads the input text sequence and generates a second representation of the input text sequence, and 
 a decoder which sequentially predicts a next term of the canonical form, based on the first and second representations and a predicted prefix of the canonical form, the prefix containing a sequence of at least one word; 
 
 receiving an input text sequence, containing a sequence of words; 
 with a processor, predicting a canonical form, containing a sequence of words, for the input text sequence with the trained neural network model; and 
 outputting information based on the predicted canonical form. 
 
     
     
       2. The method of  claim 1 , further comprising:
 parsing the predicting canonical form to generate a logical form; and 
 generating a query based on the logical form. 
 
     
     
       3. The method of  claim 2 , further comprising:
 querying a knowledge base with the query; and 
 retrieving a response to the query from the knowledge base, the output information being based on the response. 
 
     
     
       4. The method of  claim 1 , further comprising:
 training the neural network model on training data, the training data comprising training pairs, each training pair including a canonical form and a corresponding text sequence. 
 
     
     
       5. The method of  claim 4 , further comprising:
 generating the training pairs comprising collecting text sequences for a set of canonical forms using crowdsourcing. 
 
     
     
       6. The method of  claim 1 , wherein the first neural network comprises a first recurrent neural network. 
     
     
       7. The method of  claim 6 , wherein the first recurrent neural network is a first long short-term memory neural network. 
     
     
       8. The method of  claim 1 , wherein the decoder comprises a second neural network which sequentially predicts the prefix of the canonical form. 
     
     
       9. The method of  claim 8 , wherein the second neural network comprises a second long short-term memory neural network. 
     
     
       10. The method of  claim 8 , wherein the decoder comprises a multilayer perceptron which sequentially generates a next term of the canonical form based on the first and second representations and the predicted prefix of the canonical form. 
     
     
       11. The method of  claim 1 , wherein the next term of the canonical form is estimated as:
     P ( y   t   |u   l   ,u   b   ,c   l,t-1 )= s ′( W′   2 ( s ′( W′   1 ( z ))))  (3),
 
 where z is a combined representation generated from the first and second representations and the predicted prefix c l,t-1  of the canonical form, W′ 1 , W′ 2  are parameter matrices that are learned during training, and s′ is a non-linear activation function. 
 
     
     
       12. The method of  claim 1 , wherein the neural network model includes at least one embedding layer which converts words of the input text sequence to vectorial representations. 
     
     
       13. A system comprising memory which stores instructions for performing the method of  claim 1  and a processor in communication with the memory which executes the instructions. 
     
     
       14. A computer program product comprising a non-transitory storage medium storing instructions which, when executed by a computer, perform the method of  claim 1 . 
     
     
       15. A method comprising:
 providing a neural network model which has been trained to predict a canonical form for an input text sequence, the neural network model comprising:
 an encoder which comprises a first multilayer perceptron which generates a first representation of the input text sequence based on a representation of n-grams in the text sequence, and 
 a first recurrent neural network which reads the input text sequence and generates a second representation of the input text sequence, and 
 a decoder which sequentially predicts a next term of the canonical form, based on the first and second representations and a predicted prefix of the canonical form; 
 
 receiving an input text sequence; 
 with a processor, predicting a canonical form for the input text sequence with the trained neural network model; and 
 outputting information based on the predicted canonical form. 
 
     
     
       16. A system comprising:
 memory which stores a neural network model which has been trained to predict a canonical form for an input text sequence, the neural network model comprising:
 an encoder which generates a first representation of the input text sequence based on a representation of n-grams in the text sequence and a second representation of the input text sequence generated by a first neural network, and 
 a decoder which sequentially predicts terms of the canonical form based on the first and second representations and a predicted prefix of the canonical form; 
 
 a prediction component which predicts a canonical form for an input text sequence with the trained neural network model; 
 a semantic parser which generates a logical form based on the predicted canonical form; 
 an output component which outputs information based on the predicted canonical form; and 
 a processor which implements the prediction component and the output component. 
 
     
     
       17. The system of  claim 16 , further comprising a learning component which trains the neural network model on training data, the training data comprising training pairs, each training pair including a canonical form and a corresponding text sequence. 
     
     
       18. The system of  claim 17 , further comprising a querying component which queries a knowledge base with a query based on the logical form for retrieving responsive information. 
     
     
       19. A method for predicting a canonical form comprising:
 providing training data, the training data comprising a collection of training pairs, each training pair in the collection including a canonical form, containing a sequence of words, and a corresponding text sequence, containing a sequence of words; 
 with the training data, training a neural network model to predict a canonical form, containing a sequence of words, for an input text sequence, the neural network model comprising:
 an encoder which generates a first representation of the input text sequence based on a representation of n-grams in the text sequence and a second representation of the input text sequence generated by a first neural network, and 
 a decoder which sequentially predicts terms of the canonical form based on the first and second representations and a predicted prefix of the canonical form, each of the terms of the canonical form including at least one word; 
 
 receiving an input text sequence, containing a sequence of words; 
 with a processor, predicting a canonical form, containing a sequence of words, for the input text sequence with the trained neural network model; and 
 outputting information based on the predicted canonical form.