System and method for determining incoming observation to be out-of-distribution (OOD) sample during neural network (NN) inference

Abstract

A system may include at least one processor configured to: obtain a trained neural network (NN); obtain or calculate at least one average feature information associated with the trained NN, each of the at least one average feature information including a given average feature information summarizing in-class statistics that each layer of the trained NN uses for a given class; receive an incoming observation influencing a given layer; calculate a corresponding feature information of the incoming observation, the corresponding feature information summarizing statistics of the incoming observation for the given layer; classify the incoming observation as being in the given class; calculate a distance score associated with a distance between the corresponding feature information of the incoming observation and the given average feature information; determine the incoming observation is an out-of-distribution (OOD) sample; and output an alert indicating the incoming observation is OOD and/or discard the incoming observation's classification.

What is claimed is: 
     
         1 . A system, comprising:
 at least one processor configured to:
 obtain a trained neural network (NN) having classes, layers, and activation weights for the classes and layers, the classes including a given class, the layers including a given layer; 
 obtain or calculate at least one average feature information associated with the trained NN, each of the at least one average feature information including a given average feature information summarizing in-class statistics that each layer of the layers of the trained NN uses for the given class, wherein the at least one average feature information is at least one average topological persistence diagram, wherein the given average feature information is a given average topological persistence diagram; 
   receive an incoming observation influencing the given layer;   calculate a corresponding feature information of the incoming observation, the corresponding feature information summarizing statistics of the incoming observation for the given layer, wherein the corresponding feature information is a corresponding topological persistence diagram;   based at least on the trained NN, classify the incoming observation as being in the given class;   for the incoming observation classified to be in the given class, calculate a distance score associated with a distance between the corresponding feature information of the incoming observation and the given average feature information;   based at least on the calculated distance score, determine that the incoming observation is an out-of-distribution (OOD) sample; and   upon a determination that the incoming observation is the OOD sample, at least one of output an alert indicating that the incoming observation is OOD or discard the classification of the incoming observation as being in the given class.   
     
     
         2 . The system of  claim 1 , wherein the trained NN is a trained convolutional neural network (CNN), wherein the layers are convolutional layers, the given layer is a given convolutional layer, wherein the at least one processor is further configured to: downsample rasterized inputs and rasterized outputs along at least one of (a) spatial coordinates of an image obtained from the given convolutional layer or (b) channels of the given convolutional layer to generate a bipartite graph, the bipartite graph having the activation weights connecting the downsampled rasterized inputs to the downsampled rasterized outputs of the given layer, the bipartite graph used for topological persistence diagram calculations. 
     
     
         3 . The system of  claim 2 , wherein the at least one processor is further configured to: downsample the rasterized inputs and the rasterized outputs along the spatial coordinates of the image obtained from the given convolutional layer. 
     
     
         4 . The system of  claim 2 , wherein the at least one processor is further configured to: downsample the rasterized inputs and the rasterized outputs along the channels of the given convolutional layer. 
     
     
         5 . The system of  claim 2 , wherein the at least one processor is further configured to: downsample the rasterized inputs and the rasterized outputs along the spatial coordinates of the image obtained from the given convolutional layer and along the channels of the given convolutional layer. 
     
     
         6 . The system of  claim 1 , wherein the trained NN is at least one of a trained deep neural network (DNN) or a trained convolutional neural network (CNN). 
     
     
         7 . The system of  claim 1 , wherein the at least one processor is further configured to: based at least on the calculated distance score and a predetermined distance threshold, determine that the incoming observation is an out-of-distribution (OOD) sample. 
     
     
         8 . The system of  claim 1 , wherein the at least one processor is further configured to: upon the determination that the incoming observation is the OOD sample, output the alert indicating that the incoming observation is OOD. 
     
     
         9 . The system of  claim 1 , wherein the at least one processor is further configured to: upon the determination that the incoming observation is the OOD sample, discard the classification of the incoming observation as being in the given class. 
     
     
         10 . The system of  claim 1 , wherein the at least one processor is further configured to: use the trained NN to perform automatic target recognition (ATR). 
     
     
         11 . The system of  claim 1 , wherein the at least one processor is further configured to at least one of: use the trained NN to perform a computer-vision-based autonomous landing; use the trained NN to perform computer-vision-based collision avoidance; use the trained NN to perform detection of activity of a person onboard a vehicle; or use the trained NN to perform computer-vision-based vehicle path planning. 
     
     
         12 . The system of  claim 1 , wherein the system is a vehicle. 
     
     
         13 . A method, comprising:
 obtaining, by at least one processor, a trained neural network (NN) having classes, layers, and activation weights for the classes and layers, the classes including a given class, the layers including a given layer;   obtaining or calculating, by the at least one processor, at least one average feature information associated with the trained NN, each of the at least one average feature information including a given average feature information summarizing in-class statistics that each layer of the layers of the trained NN uses for the given class, wherein the at least one average feature information is at least one average topological persistence diagram, wherein the given average feature information is a given average topological persistence diagram;   receiving, by the at least one processor, an incoming observation influencing the given layer;   calculating, by the at least one processor, a corresponding feature information of the incoming observation, the corresponding feature information summarizing statistics of the incoming observation for the given layer, wherein the corresponding feature information is a corresponding topological persistence diagram;   based at least on the trained NN, classifying, by the at least one processor, the incoming observation as being in the given class;   for the incoming observation classified to be in the given class, calculating, by the at least one processor, a distance score associated with a distance between the corresponding feature information of the incoming observation and the given average feature information;   based at least on the calculated distance score, determining, by the at least one processor, that the incoming observation is an out-of-distribution (OOD) sample; and   upon a determination that the incoming observation is the OOD sample, at least one of: outputting, by the at least one processor, an alert indicating that the incoming observation is OOD; or discarding, by the at least one processor, the classification of the incoming observation as being in the given class.   
     
     
         14 . The method of  claim 13 , wherein the trained NN is a trained convolutional neural network (CNN), wherein the layers are convolutional layers, the given layer is a given convolutional layer, wherein the method further comprises: downsampling, by the at least one processor, rasterized inputs and rasterized outputs along at least one of (a) spatial coordinates of an image obtained from the given convolutional layer or (b) channels of the given convolutional layer to generate a bipartite graph, the bipartite graph having the activation weights connecting the downsampled rasterized inputs to the downsampled rasterized outputs of the given layer, the bipartite graph used for topological persistence diagram calculations. 
     
     
         15 . The system of  claim 1 , wherein the at least one processor is at least two processors.