Systems and methods for intelligently curating machine learning training data and improving machine learning model performance

Abstract

Systems and methods of intelligent formation and acquisition of machine learning training data for implementing an artificially intelligent dialogue system includes constructing a corpora of machine learning test corpus that comprise a plurality of historical queries and commands sampled from production logs of a deployed dialogue system; configuring training data sourcing parameters to source a corpora of raw machine learning training data from remote sources of machine learning training data; calculating efficacy metrics of the corpora of raw machine learning training data, wherein calculating the efficacy metrics includes calculating one or more of a coverage metric value and a diversity metric value of the corpora of raw machine learning training data; using the corpora of raw machine learning training data to train the at least one machine learning classifier if the calculated coverage metric value of the corpora of machine learning training data satisfies a minimum coverage metric threshold.

What is claimed: 
     
         1 . A system for intelligently identifying machine learning training data for implementing a machine learning-based dialogue service, the system comprising:
 one or more sources of machine learning training data;   one or more hardware computing servers implementing a machine learning-based dialogue service that:
 calculates, using the one or more hardware computing servers, one or more efficacy metrics of a corpora of raw machine learning training data; and 
 identifies whether to train at least one machine learning classifier of the machine learning-based dialogue system based on the one or more efficacy metrics of the corpora of raw machine learning training data.