Systems and methods for machine learning based multi intent segmentation and classification

Abstract

Systems and methods for synthesizing training data for multi-intent utterance segmentation include identifying a first corpus of utterances comprising a plurality of distinct single-intent in-domain utterances; identifying a second corpus of utterances comprising a plurality of distinct single-intent out-of-domain utterances; identifying a third corpus comprising a plurality of distinct conjunction terms; forming a multi-intent training corpus comprising synthetic multi-intent utterances, wherein forming each distinct multi-intent utterance includes: selecting a first distinct in-domain utterance from the first corpus of utterances; probabilistically selecting one of a first out-of-domain utterance from the second corpus and a second in-domain utterance from the first corpus; probabilistically selecting or not selecting a distinct conjunction term from the third corpus; and forming a synthetic multi-intent utterance including appending the first in-domain utterance with one of the first out-of-domain utterance from the second corpus of utterances and the second in-domain utterance from the first corpus of utterances.

What is claimed is: 
     
         1 . A method for synthesizing training data for multi-intent utterance segmentation within a machine learning-based dialogue system, the method comprising:
 identifying a first corpus of utterances comprising a plurality of distinct single-intent in-domain utterances;   identifying a second corpus of utterances comprising a plurality of distinct single-intent out-of-domain utterances;   identifying a third corpus comprising a plurality of distinct conjunction terms;   forming, by the machine learning-based dialogue system, a multi-intent training corpus comprising synthetic multi-intent utterances, wherein forming each distinct multi-intent utterance of the multi-intent training corpus includes:
 selecting a first distinct in-domain utterance from the plurality of distinct single-intent in-domain utterances of the first corpus of utterances; 
 probabilistically selecting one of a first out-of-domain utterance from the second corpus of utterances and a second in-domain utterance from the first corpus of utterances; 
 probabilistically selecting or not selecting a distinct conjunction term from the third corpus of conjunction terms; and 
 forming a synthetic multi-intent utterance including appending the first in-domain utterance with one of the first out-of-domain utterance from the second corpus of utterances and the second in-domain utterance from the first corpus of utterances. 
   
     
     
         2 . The method according to  claim 1 , further comprising:
 identifying a conjunction-inclusion probability that a conjunction term would be appended to the first distinct in-domain utterance; and   if the conjunction-inclusion probability satisfies or exceeds a conjunction-inclusion threshold, randomly selecting a distinct conjunction term from the plurality of distinct conjunction terms of the third corpus.   
     
     
         3 . The method according to  claim 1 , further comprising:
 identifying an out-of-domain-inclusion probability that an out-of-domain utterance would be appended to the first distinct in-domain utterance, wherein if the out-of-domain-inclusion probability satisfies or exceeds an out-of-domain-inclusion threshold, randomly selecting a first distinct out-of-domain utterance from the plurality of distinct single-intent out-of-domain utterances of the second corpus of utterances.   
     
     
         4 . The method according to  claim 3 , further comprising:
 in response to selecting the first distinct out-of-domain utterance, concatenating the distinct conjunction term to a boundary of the first in-domain utterance and concatenating the first distinct out-of-domain utterance after the distinct conjunction term.   
     
     
         5 . The method according to  claim 1 , further comprising:
 identifying an out-of-domain-inclusion probability that an out-of-domain utterance would be appended to the first distinct in-domain utterance, wherein if the out-of-domain-inclusion probability does not satisfy the out-of-domain-inclusion threshold, randomly selecting a second distinct in-domain utterance from the plurality of distinct single-intent in-domain utterances of the first corpus of utterances.   
     
     
         6 . The method according to  claim 1 , further comprising:
 identifying a conjunction-inclusion probability that a conjunction term would be appended to the first distinct in-domain utterance;   if the conjunction-inclusion probability satisfies or exceeds a conjunction-inclusion threshold, randomly selecting a distinct conjunction term from the plurality of distinct conjunction terms of the third corpus;   identifying an out-of-domain-inclusion probability that an out-of-domain utterance would be appended to the first distinct in-domain utterance, wherein:
 (i) if the out-of-domain-inclusion probability satisfies or exceeds an out-of-domain-inclusion threshold, randomly selecting a first distinct out-of-domain utterance from the plurality of distinct single-intent out-of-domain utterances of the second corpus of utterances, or 
 (ii) if the out-of-domain-inclusion probability does not satisfy the out-of-domain-inclusion threshold, randomly selecting a second distinct in-domain utterance from the plurality of distinct single-intent in-domain utterances of the first corpus of utterances. 
   
     
     
         7 . The method according to  claim 1 , wherein:
 each of the plurality of distinct single-intent in-domain utterances of the first corpus comprise a single-intent in-domain utterance, and   each of the plurality of distinct single-intent out-of-domain utterances of the second corpus comprise a single-intent out-of-domain utterance.   
     
     
         8 . The method according to  claim 1 , further comprising:
 training a span-predicting utterance segmentation model using the multi-intent training corpus, wherein the span-predicting utterance segmentation model classifies each distinct utterance span of a subject multi-intent utterance that forms a complete semantic expression within the subject multi-intent utterance.   
     
     
         9 . The method according to  claim 8 , further comprising:
 receiving an input multi-intent utterance at the machine learning-based dialogue system;   predicting two or more boundary classification labels for two or more distinct tokens of the input multi-intent utterance; and   segmenting, at two or more boundary classification labels, the input multi-intent utterance into two or more distinct single-intent utterance components.   
     
     
         10 . The method according to  claim 9 , further comprising:
 providing each of the two or more distinct single-intent utterance components to one of a plurality of concurrently operating distinct single-intent machine learning classifiers; and   generating by each respective one of the plurality of concurrently operating distinct machine learning classifiers an intent classification label for each of the two or more distinct single-intent utterance components.   
     
     
         11 . The method according to  claim 1 , further comprising:
 training a joint model using the multi-intent training corpus comprising synthetic multi-intent utterances, wherein the joint model perform multiple distinct machine learning tasks, the joint model comprising an intent machine learning classifier that predicts an intent label for a target utterance and a slot segment machine learning model that predicts a slot label that identifies a semantic concept of a given segment of the target utterance.   
     
     
         12 . The method according to  claim 11 , further comprising:
 receiving an input multi-intent utterance; and   identifying whether the input multi-intent utterance is an entangled multi-intent utterance based on an entanglement threshold, wherein an entangled multi-intent utterance relates to a subject multi-intent utterance in which two or more distinct intents within the subject multi-intent utterance cannot be disintegrated with ease and satisfy or exceed an entanglement threshold.   
     
     
         13 . The method according to  claim 12 , wherein:
 if the input multi-intent utterance comprises the entangled multi-intent utterance, providing the entangled multi-intent utterance as input into the joint model;   at the joint model, predicting an intent classification label and a slot value classification label for each identified token of the entangled multi-intent utterance.   
     
     
         14 . The method according to  claim 1 , further comprising:
 training a joint model with segmentation using the multi-intent training corpus comprising synthetic multi-intent utterances, where the joint model with segmentation performs multi-distinct machine learning tasks, the joint model with segmentation including (i) a combination of a segmentation model, (ii) an intent classification model, and (iii) a slot value classification model.   
     
     
         15 . The method according to  claim 14 , further comprising:
 receiving an input multi-intent utterance; and   identifying whether the input multi-intent utterance comprises a long, multi-intent utterance based on an aggregated span threshold, wherein the long, multi-intent utterance relates to a subject multi-intent utterance in which an aggregate of multiple distinct utterance spans of the subject multi-intent utterance satisfies or exceeds an aggregated span threshold.   
     
     
         16 . The method according to  claim 15 , wherein:
 if the input multi-intent utterance comprises the long multi-intent utterance, providing the long multi-intent utterance as input into the joint model with segmentation;   at the joint model with segmentation, (i) predicting two or more boundary classification labels for two or more distinct tokens of the long multi-intent utterance, (ii) predicting an intent classification label and (iii) a slot value classification label for each identified token of the long multi-intent utterance.   
     
     
         17 . A method for synthesizing training data for multi-intent utterance segmentation within a single-intent machine learning-based dialogue system, the method comprising:
 sourcing a first corpus of utterances comprising a plurality of distinct single-intent in-domain utterances;   sourcing a second corpus of utterances comprising a plurality of distinct single-intent out-of-domain utterances;   sourcing a third corpus comprising a plurality of distinct conjunction terms;   constructing, by the machine learning-based dialogue system, a multi-intent training corpus comprising synthetic multi-intent utterances, wherein forming each distinct multi-intent utterance of the multi-intent training corpus includes:
 selecting a first distinct in-domain utterance from the plurality of distinct single-intent in-domain utterances of the first corpus of utterances; 
 probabilistically selecting one of a first out-of-domain utterance from the second corpus of utterances and a second in-domain utterance from the first corpus of utterances; 
 probabilistically selecting or not selecting a distinct conjunction term from the third corpus of conjunction terms; and 
 constructing a synthetic multi-intent utterance including appending the first in-domain utterance with one of the first out-of-domain utterance from the second corpus of utterances and the second in-domain utterance from the first corpus of utterances. 
   
     
     
         18 . The method according to  claim 1 , further comprising:
 computing, by the machine learning-based system, a conjunction-inclusion probability that a conjunction term would be appended to the first distinct in-domain utterance;   if the conjunction-inclusion probability satisfies or exceeds a conjunction-inclusion threshold, randomly selecting a distinct conjunction term from the plurality of distinct conjunction terms of the third corpus;   computing, by the machine learning-based system, an out-of-domain-inclusion probability that an out-of-domain utterance would be appended to the first distinct in-domain utterance, wherein:
 (i) if the out-of-domain-inclusion probability satisfies or exceeds an out-of-domain-inclusion threshold, randomly selecting a first distinct out-of-domain utterance from the plurality of distinct single-intent out-of-domain utterances of the second corpus of utterances, or 
 (ii) if the out-of-domain-inclusion probability does not satisfy the out-of-domain-inclusion threshold, randomly selecting a second distinct in-domain utterance from the plurality of distinct single-intent in-domain utterances of the first corpus of utterances. 
   
     
     
         19 . A system for intelligently synthesizing training data for multi-intent utterance segmentation within a machine learning-based dialogue system, the system comprising:
 a datastore comprising:
 a first corpus of utterances comprising a plurality of distinct single-intent in-domain utterances; 
 a second corpus of utterances comprising a plurality of distinct single-intent out-of-domain utterances; 
 a third corpus comprising a plurality of distinct conjunction terms; 
   a machine learning-based dialogue system being implemented by a distributed network of computers includes:
 a training data synthesis module that:
 constructs a multi-intent training corpus comprising synthetic multi-intent utterances, wherein forming each distinct multi-intent utterance of the multi-intent training corpus includes: 
 selects a first distinct in-domain utterance from the plurality of distinct single-intent in-domain utterances of the first corpus of utterances; 
 probabilistically selects one of a first out-of-domain utterance from the second corpus of utterances and a second in-domain utterance from the first corpus of utterances; 
 probabilistically selects or not selects a distinct conjunction term from the third corpus of conjunction terms; and 
 constructs a synthetic multi-intent utterance including appending the first in-domain utterance with one of the first out-of-domain utterance from the second corpus of utterances and the second in-domain utterance from the first corpus of utterances.