System and method for augmenting training data for natural language to meaning representation language systems

Abstract

Techniques for augmenting training data include accessing training data comprising a plurality of training examples comprising a first training example comprising a first natural language utterance and a first logical form for the first natural language utterance. A second natural language utterance is generated by adding or replacing one or more values in the first natural language utterance. A logical form for the second natural language utterance is generated. A second training example is generated, comprising the second natural language utterance and the logical form for the second natural language utterance. The training data is augmented by adding the second training example to the plurality of training examples to generate an augmented training data set. A machine learning model is trained to generate logical forms for utterances using the augmented training data set.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 (a) accessing training data comprising a plurality of training examples comprising a first training example, the first training example comprising a first natural language utterance, a first logical form for the first natural language utterance, and first metadata associated with the first natural language utterance, the first metadata including information about a database schema for a database to be queried using a logical form;   (b) generating a second natural language utterance by adding or replacing one or more values in the first natural language utterance;   (c) generating the logical form for the second natural language utterance;   (d) producing updated metadata based on the first metadata and the second natural language utterance;   (e) generating a second training example comprising the second natural language utterance, the logical form for the second natural language utterance, and the updated metadata;   (f) augmenting the training data by adding the second training example to the plurality of training examples to generate an augmented training data set; and   (g) training a machine learning model to generate logical forms for utterances using the augmented training data set.   
     
     
         2 . The computer-implemented method of  claim 1 , further comprising:
 repeating steps (b)-(f) to generate and add a configured number of additional training examples to the augmented training data set, wherein a type of augmentation and a set of replacement values are further configured.   
     
     
         3 . The computer-implemented method of  claim 2 , wherein the replacement values are selected, based on the configuration, by randomly generating data points. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein producing the updated metadata comprises:
 adjusting an offset value for schema linking to reflect the one or more replacement values.   
     
     
         5 . The computer-implemented method of  claim 1 , wherein the logical forms correspond to database query representations. 
     
     
         6 . The computer-implemented method of  claim 1 , further comprising:
 deploying the machine learning model to generate an output logical form for an input natural language utterance.   
     
     
         7 . A system comprising:
 one or more processors; and   one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the system to perform operations comprising:   (a) accessing training data comprising a plurality of training examples comprising a first training example, the first training example comprising a first natural language utterance, a first logical form for the first natural language utterance, and first metadata associated with the first natural language utterance, the first metadata including information about a database schema for a database to be queried using a logical form;   (b) generating a second natural language utterance by adding or replacing one or more values in the first natural language utterance;   (c) generating the logical form for the second natural language utterance;   (d) producing updated metadata based on the first metadata and the second natural language utterance;   (e) generating a second training example comprising the second natural language utterance, the logical form for the second natural language utterance;   (f) augmenting the training data by adding the second training example to the plurality of training examples to generate an augmented training data set, and the updated metadata; and   (g) training a machine learning model to generate logical forms for utterances using the augmented training data set.   
     
     
         8 . The system of  claim 7 , the operations further comprising:
 repeating steps (b)-(f) to generate and add a configured number of additional training examples to the augmented training data set, wherein a type of augmentation and a set of replacement values are further configured.   
     
     
         9 . The system of  claim 8 , wherein the replacement values are selected, based on the configuration, by randomly generating data points. 
     
     
         10 . The system of  claim 7 , wherein producing the updated metadata comprises:
 adjusting an offset value for schema linking to reflect the one or more replacement values.   
     
     
         11 . The system of  claim 7 , wherein the logical forms correspond to database query representations. 
     
     
         12 . The system of  claim 7 , further comprising:
 deploying the machine learning model to generate an output logical form for an input natural language utterance.   
     
     
         13 . One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause a system to perform operations comprising:
 (a) accessing training data comprising a plurality of training examples comprising a first training example, the first training example comprising a first natural language utterance, and a first logical form for the first natural language utterance, and first metadata associated with the first natural language utterance, the first metadata including information about a database schema for a database to be queried using a logical form;   (b) generating a second natural language utterance by adding or replacing one or more values in the first natural language utterance;   (c) generating the logical form for the second natural language utterance;   (d) producing updated metadata based on the first metadata and the second natural language utterance;   (e) generating a second training example comprising the second natural language utterance, the logical form for the second natural language utterance, and the updated metadata;   (f) augmenting the training data by adding the second training example to the plurality of training examples to generate an augmented training data set; and   (g) training a machine learning model to generate logical forms for utterances using the augmented training data set.   
     
     
         14 . The one or more non-transitory computer-readable media of  claim 13 , the operations further comprising:
 repeating steps (b)-(f) to generate and add a configured number of additional training examples to the augmented training data set, wherein a type of augmentation and a set of replacement values are further configured.   
     
     
         15 . The one or more non-transitory computer-readable media of  claim 13 , wherein producing the updated metadata comprises:
 adjusting an offset value for schema linking to reflect the one or more replacement values.   
     
     
         16 . The one or more non-transitory computer-readable media of  claim 13 , wherein the logical forms correspond to database query representations. 
     
     
         17 . The one or more non-transitory computer-readable media of  claim 13 , the operations further comprising:
 deploying the machine learning model to generate an output logical form for an input natural language utterance.