Systems and methods for automatically detecting and repairing slot errors in machine learning training data for a machine learning-based dialogue system

Abstract

Systems and methods for automatically detecting annotation discrepancies in annotated training data samples and repairing the annotated training data samples for a machine learning-based automated dialogue system include evaluating a corpus of a plurality of distinct training data samples; identifying one or more of a slot span defect and a slot label defect of a target annotated slot span of a target training data sample of the corpus based on the evaluation; and automatically correcting one or more annotations of the target annotated slot span based on the identified one or more of the slot span defect and the slot label defect.

What is claimed: 
     
         1 . A method for automatically detecting annotation discrepancies in annotated training data samples and repairing the annotated training data samples for a machine learning-based automated dialogue system, the method comprising:
 evaluating a corpus of a plurality of distinct training data samples;   identifying one or more of a slot span defect and a slot label defect of a target annotated slot span of a target training data sample of the corpus based on the evaluation, wherein identifying whether a target annotated slot of a training data sample deviates from a left- and right n-gram set created for the corpus; and   automatically correcting one or more annotations of the target annotated slot span based on the identified one or more of the slot span defect and the slot label defect.   
     
     
         2 . A system for automatically detecting annotation discrepancies in annotated training data samples and repairing the annotated training data samples for a machine learning-based automated dialogue system, the system comprising:
 a label variation evaluator or a slot label evaluator implemented by one or more computers that:
 evaluate a corpus of a plurality of distinct training data samples; 
 identify one or more of a slot span defect and a slot label defect of a target annotated slot span of a target training data sample of the corpus based on the evaluation, wherein identifying whether a target annotated slot of a training data sample deviates from a left- and right n-gram set created for the corpus; and 
 automatically correct one or more annotations of the target annotated slot span based on the identified one or more of the slot span defect and the slot label defect.