Label induction

Abstract

Systems and methods for document classification are described. Embodiments of the present disclosure generate classification data for a plurality of samples using a neural network trained to identify a plurality of known classes; select a set of samples for annotation from the plurality of samples using an open-set metric based on the classification data, wherein the annotation includes an unknown class; and train the neural network to identify the unknown class based on the annotation of the set of samples.

What is claimed is: 
     
         1 . A method comprising:
 receiving an electronic document;   classifying the electronic document using a neural network to obtain classification data, wherein the neural network is trained by iteratively selecting samples for annotation with an unknown class using an open-set metric based on predicted classification data; and   displaying the electronic document via a customized user interface based on the classification data.   
     
     
         2 . The method of  claim 1 , further comprising:
 generating the predicted classification data for a plurality of samples using the neural network, wherein the neural network is trained to identify a plurality of known classes;   selecting a set of samples for annotation with the unknown class from the plurality of samples using the open-set metric based on the predicted classification data; and   training the neural network to identify the unknown class based on the annotation of the set of samples.   
     
     
         3 . The method of  claim 1 , further comprising:
 identifying a document type based on the classification data; and   selecting an interface element associated with the document type, wherein the customized user interface includes the interface element.   
     
     
         4 . A method comprising:
 generating classification data for a plurality of samples using a neural network trained to identify a plurality of known classes;   selecting a set of samples for annotation from the plurality of samples using an open-set metric based on the classification data, wherein the annotation includes an unknown class; and   training the neural network to identify the unknown class based on the annotation of the set of samples.   
     
     
         5 . The method of  claim 4 , further comprising:
 computing a multi-label loss based on the classification data and ground truth labels, wherein each of the ground truth labels describes a known class of the plurality of known classes; and   updating parameters of the neural network based on the multi-label loss.   
     
     
         6 . The method of  claim 4 , further comprising:
 generating a feature embedding corresponding to each of the plurality of samples using the neural network, wherein the classification data is generated based on the feature embedding.   
     
     
         7 . The method of  claim 4 , wherein:
 the classification data includes prediction logits, uncertainty measures, or both.   
     
     
         8 . The method of  claim 4 , wherein:
 the open-set metric indicates that the plurality of known classes do not characterize the set of samples to a threshold level.   
     
     
         9 . The method of  claim 4 , further comprising:
 clustering the plurality of samples to obtain a plurality of clusters; and   selecting a cluster from the plurality of clusters based on the open-set metric, wherein the set of samples includes samples from the cluster.   
     
     
         10 . The method of  claim 9 , further comprising:
 computing an activation logit value for each sample in the selected cluster; and   identifying a maximum activation logit value based on the activation logit value for each sample in the selected cluster, wherein the open-set metric is based on the maximum activation logit value.   
     
     
         11 . The method of  claim 10 , wherein:
 the selected cluster minimizes the maximum activation logit value.   
     
     
         12 . The method of  claim 4 , further comprising:
 identifying a class of the plurality of known classes;   excluding the class from the plurality of known classes to obtain a reduced set of known classes; and   computing the open-set metric based on the reduced set of known classes.   
     
     
         13 . The method of  claim 4 , further comprising:
 displaying the set of samples in an annotation interface; and   receiving annotation input via the annotation interface, wherein the training is based on the annotation input.   
     
     
         14 . The method of  claim 13 , further comprising:
 identifying a shared label of the set of samples based on the annotation input, wherein the annotation is based on the shared label.   
     
     
         15 . The method of  claim 13 , further comprising:
 identifying a distinguishing label of the set of samples based on the annotation input, wherein the annotation is based on the distinguishing label.   
     
     
         16 . An apparatus comprising:
 a processor;   a memory including instructions executable by the processor;   a neural network trained to identify a plurality of known classes for a plurality of samples;   a clustering component configured to cluster the plurality of samples to obtain a plurality of clusters; and   an open-set metric component configured to identify a cluster of the plurality of clusters for annotation based on an open-set metric.   
     
     
         17 . The apparatus of  claim 16 , further comprising:
 a training component configured to train the neural network based on the annotation.   
     
     
         18 . The apparatus of  claim 16 , further comprising:
 an annotation component configured to display samples of the identified cluster in an annotation interface.   
     
     
         19 . The apparatus of  claim 16 , further comprising:
 a user interface component configured to display a customized user interface based on classification data generated by the neural network.   
     
     
         20 . The apparatus of  claim 16 , wherein:
 the neural network includes a transformer network that includes a classification head.