Hamming distance based robust output encoding for improved generalization

Abstract

A computer-implemented method generates a hamming code based target label for each class of a dataset in which hamming distance between the target labels in the dataset is maximized and trains a convolutional neural network with the hamming codes based target label to thereby produce a trained AI model. The confusability between classes of the dataset is determined using a confusion matrix. The hamming distances of classes of the dataset that are determined to be more confusable are set to higher values than the hamming distances of classes of the dataset that are determined to be less confusable.

What is claimed is: 
     
       1. A computer-implemented method comprising:
 generating a hamming code based target label for each class of a dataset in which hamming distance between the target labels in the dataset is maximized; and 
 training a convolutional neural network with the modified dataset to thereby produce a trained AI model, 
 wherein confusability between classes of the dataset is determined using a confusion matrix, and 
 wherein hamming distances of classes of the dataset that are determined to be more confusable are set to higher values than the hamming distances of classes of the dataset that are determined to be less confusable. 
 
     
     
       2. The computer-implemented method of  claim 1 , wherein the confusability between classes of the dataset is determined by reconstructing data for each class using an autoencoder trained using a first class of the classes and determining a reconstruction error for each class other than the first class. 
     
     
       3. The computer-implemented method of  claim 1 , wherein the confusability between classes of the dataset is determined by using a data similarity method to compute a confusion matrix. 
     
     
       4. The computer-implemented method of  claim 1 , wherein hamming codes are generated by maximizing the hamming distance between target labels; where the target labels are weighted based on the confusability between classes in the dataset, and wherein there is one hamming code per class in the dataset. 
     
     
       5. The computer-implemented method of  claim 1 , wherein training is conducted by backpropagation with binary cross entropy loss. 
     
     
       6. The computer-implemented method of  claim 1 , further comprising:
 forward propagating using the trained AI model; 
 binarizing top-k logits of a sigmoid layer to produce a binarized code; 
 computing Euclidean distance from the binarized code to a set of target hamming codes; and 
 selecting an output label corresponding to a closest hamming code to thereby produce an inference result. 
 
     
     
       7. The computer-implemented method of  claim 1 , wherein the dataset is a dataset of images. 
     
     
       8. A non-transitory computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform a method comprising: generating a hamming code based target label for each class of a dataset in which hamming distance between the target labels in the dataset is maximized; and training a convolutional neural network with the modified dataset to thereby produce a trained AI model, wherein confusability between classes of the dataset is determined using a confusion matrix, and wherein the hamming distances of classes of the dataset that are determined to be more confusable are set to higher values than the hamming distances of classes of the dataset that are determined to be less confusable. 
     
     
       9. The non-transitory computer program product of  claim 8 , wherein the confusability between classes of the dataset is determined by reconstructing data for each class using an autoencoder trained using a first class of the classes and determining a reconstruction error for each class other than the first class. 
     
     
       10. The non-transitory computer program product of  claim 8 , wherein the confusability between classes of the dataset is determined by using a data similarity method to compute a confusion matrix. 
     
     
       11. The non-transitory computer program product of  claim 8 , wherein hamming codes are generated by maximizing the hamming distance between target labels; where the target labels are weighted based on the confusability between classes in the dataset, and wherein there is one hamming code per class in the dataset. 
     
     
       12. The non-transitory computer program product of  claim 8 , wherein training is conducted by backpropagation with binary cross entropy loss. 
     
     
       13. The non-transitory computer program product of  claim 8 , forward propagating using the trained AI model; binarizing top-k logits of a sigmoid layer to produce a binarized code; computing Euclidean distance from the binarized code to a set of target hamming codes; and selecting an output label corresponding to a closest hamming code to thereby produce an inference result. 
     
     
       14. The non-transitory computer program product of  claim 8 , wherein the dataset is a dataset of images. 
     
     
       15. A system including one or more processors configured to implement a method comprising:
 generating a hamming code based target label for each class of a dataset in which hamming distance between the target labels in the dataset is maximized; and 
 training a convolutional neural network with the modified dataset to thereby produce a trained AI model, 
 wherein confusability between classes of the dataset is determined using a confusion matrix, and 
 wherein hamming distances of classes of the dataset that are determined to be more confusable are set to higher values than the hamming distances of classes of the dataset that are determined to be less confusable. 
 
     
     
       16. The system of  claim 15 , wherein the confusability between classes of the dataset is determined by reconstructing data for each class using an autoencoder trained using a first class of the classes and determining a reconstruction error for each class other than the first class. 
     
     
       17. The system of  claim 15 , wherein the confusability between classes of the dataset is determined by using a data similarity method to compute a confusion matrix. 
     
     
       18. The system of  claim 15 , wherein hamming codes are generated by maximizing the hamming distance between target labels; where the target labels are weighted based on the confusability between classes in the dataset, and wherein there is one hamming code per class in the dataset. 
     
     
       19. The system of  claim 15 , wherein training is conducted by backpropagation with binary cross entropy loss. 
     
     
       20. The system of  claim 15 , wherein the method further comprising:
 forward propagating using the trained AI model; 
 binarizing top-k logits of a sigmoid layer to produce a binarized code; 
 computing Euclidean distance from the binarized code to a set of target hamming codes; and 
 selecting an output label corresponding to a closest hamming code to thereby produce an inference result.