Method and system for training virtual agents through fallback analysis

Abstract

A method and system for training a virtual agent through fallback analysis is provided herein. The method comprises obtaining a plurality of fallback utterances. The method further comprises classifying the plurality of fallback utterances into one or more of existing intent categories, via a Machine Learning (ML) model. The method further comprises upon unsuccessful classification of one or more utterances of the plurality of fallback utterances, clustering the one or more utterances into one or more groups based on similarities among the one or more utterances, via the ML model. Further, the method comprises generating labels for the one or more groups to determine names of new intent categories associated with the one or more utterances.

What is claimed is: 
     
         1 . A computer-implemented method for training a virtual agent through fallback analysis comprising:
 obtaining a plurality of fallback utterances;   classifying the plurality of fallback utterances into one or more of existing intent categories, via a Machine Learning (ML) model;   upon unsuccessful classification of one or more utterances of the plurality of fallback utterances, clustering the one or more utterances into one or more groups based on similarities among the one or more utterances, via the ML model; and   generating labels for the one or more groups to determine names of new intent categories associated with the one or more utterances.   
     
     
         2 . The computer-implemented method of  claim 1 , further comprising training the virtual agent based on the one or more groups of the one or more utterances, wherein the one or more utterances comprises training phrases. 
     
     
         3 . The computer-implemented method of  claim 1 , wherein fallback logs comprise the plurality of fallback utterances. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein the plurality of fallback utterances corresponds to at least one of incorrect responses, ambiguous responses, or insufficient responses generated by the virtual agent for queries. 
     
     
         5 . The computer-implemented method of  claim 1 , further comprising:
 receiving a query from a user;   generating a response for the query, wherein the response is one of an incorrect response, an ambiguous response, or an insufficient response;   storing the query in the fallback logs; and   mining the logs to obtain the plurality of fallback utterances.   
     
     
         6 . The computer-implemented method of  claim 1 , wherein the ML model is a classification model. 
     
     
         7 . The computer-implemented method of  claim 6 , wherein the classification model comprises at least one of a logistic regression model, a decision tree, a random forest model, a Support Vector Machine (SVM), or an automated machine learning (AutoML) model. 
     
     
         8 . The computer-implemented method of  claim 1 , wherein the ML model is a clustering model. 
     
     
         9 . The computer-implemented method of  claim 7 , wherein the clustering model uses at least one of a K-means clustering algorithm, a hierarchical clustering algorithm, a density-based spatial clustering algorithm, a gaussian mixture model, or hybrid ensemble models. 
     
     
         10 . The computer-implemented method of  claim 1 , wherein generating the labels comprises assigning generic names to the one or more groups. 
     
     
         11 . A computer system for training a virtual agent through fallback analysis comprising, the computer system comprising: one or more computer processors, one or more computer readable memories, one or more computer readable storage devices, and program instructions stored on the one or more computer readable storage devices for execution by the one or more computer processors via the one or more computer readable memories, the program instructions comprising:
 obtaining a plurality of fallback utterances;   classifying the plurality of fallback utterances into one or more of existing intent categories, via a Machine Learning (ML) model;   upon unsuccessful classification of one or more utterances of the plurality of fallback utterances, clustering the one or more utterances into one or more groups based on similarities among the one or more utterances, via the ML model; and   generating labels for the one or more groups to determine names of new intent categories associated with the one or more utterances.   
     
     
         12 . The system of  claim 11 , further comprising training the virtual agent based on the one or more groups of the one or more utterances, wherein the one or more utterances comprises training phrases. 
     
     
         13 . The system of  claim 11 , wherein fallback logs comprise the plurality of fallback utterances, and wherein the plurality of fallback utterances corresponds to at least one of incorrect responses, ambiguous responses, or insufficient responses generated by the virtual agent for queries. 
     
     
         14 . The system of  claim 11 , further comprising:
 receiving a query from a user;   generating a response for the query, wherein the response is one of an incorrect response, an ambiguous response, or an insufficient response;   storing the query in the fallback logs; and   mining the logs to obtain the plurality of fallback utterances.   
     
     
         15 . The system of  claim 11 , wherein the ML model is a classification model. 
     
     
         16 . The system of  claim 15 , wherein the classification model comprises at least one of a logistic regression model, a decision tree, a random forest model, a Support Vector Machine (SVM), or an automated machine learning (AutoML) model. 
     
     
         17 . The system of  claim 11 , wherein the ML model is a clustering model. 
     
     
         18 . The system of  claim 17 , wherein the clustering model uses at least one of a K-means clustering algorithm, a hierarchical clustering algorithm, a density-based spatial clustering algorithm, a gaussian mixture model, or hybrid ensemble models. 
     
     
         19 . The system of  claim 11 , wherein generating the labels comprises assigning generic names to the one or more groups. 
     
     
         20 . A non-transitory computer-readable storage medium having stored thereon computer executable instruction which when executed by one or more processors, cause the one or more processors to carry out operations for training a virtual agent through fallback analysis, the operations comprising perform the operations comprising:
 obtaining a plurality of fallback utterances;   classifying the plurality of fallback utterances into one or more of existing intent categories, via a Machine Learning (ML) model;   upon unsuccessful classification of one or more utterances of the plurality of fallback utterances, clustering the one or more utterances into one or more groups based on similarities among the one or more utterances, via the ML model; and   generating labels for the one or more groups to determine names of new intent categories associated with the one or more utterances.