Method for managing training data

Abstract

Disclosed is a non-transitory computer readable medium storing a computer program. When the computer program is executed by one or more processors of a computing device, the computer program performs the following operations for processing data, and the operations may include: determining an uncertainty level with respect to labeling criteria for each of one or more data included in a dataset; determining a similarity level for one or more data included in a data subset; and selecting at least some of data included in the dataset based on the uncertainty level and the similarity level, and additionally labeling the selected data.

What is claimed is: 
     
         1 . A non-transitory computer readable medium storing a computer program, wherein when the computer program is executed by one or more processors of a computing device, the computer program performs operations for processing data, and the operations comprise:
 determining an uncertainty level with respect to labeling criteria for each of one or more data included in a dataset;   determining a similarity level for one or more data included in a data subset, wherein one or both of determining the uncertainty level and determining the similarity level are based on determining a location, in a data space, of the one or more data included in the respective one of the dataset and the data subset;   selecting at least some of data included in the dataset based on a calculation that correlates the uncertainty level and the similarity level by adding relative weights on each of the uncertainty level and the similarity level; and   labeling the selected data.   
     
     
         2 . The non-transitory computer readable medium according to  claim 1 , wherein the operations further comprise:
 mapping the one or more data included in the dataset to the solution space by inputting the one or more data included in the dataset to a classification model.   
     
     
         3 . The non-transitory computer readable medium according to  claim 1 , wherein the operations further comprise:
 after selecting the at least some of data included in the dataset based on the uncertainty level and the similarity level and labeling the selected data, updating a classification model based at least in part on the labeled data.   
     
     
         4 . The non-transitory computer readable medium according to  claim 1 , wherein selecting the at least some of data and labeling the selected data comprises:
 selectively labeling data having a high uncertainty level and a low similarity level.   
     
     
         5 . The non-transitory computer readable medium according to  claim 1 , wherein determining the uncertainty level comprises determining based at least in part on a result of classifying each of the one or more data using a classification model. 
     
     
         6 . The non-transitory computer readable medium according to  claim 5 , wherein the operations further comprise:
 mapping each of the one or more data to a solution space by inputting each of the one or more data to the classification model, and   wherein determining the uncertainty level comprises determining based on a distance between a location in a solution space of each of the one or more data and classification criteria of the classification model.   
     
     
         7 . The non-transitory computer readable medium according to  claim 1 , wherein the operations further comprise:
 mapping each of the one or more data to a data space, and   wherein determining the uncertainty level comprises determining based on a distance between a location in a data space of each of the one or more data and the classification criteria.   
     
     
         8 . The non-transitory computer readable medium according to  claim 1 , wherein the operations further comprise:
 mapping each of the one or more data included in the data subset to the solution space by inputting each of the one or more data included in the data subset to a classification model, and   wherein determining the similarity level comprises determining based on a location in the solution space of each of the one or more data included in the data subset.   
     
     
         9 . The non-transitory computer readable medium according to  claim 1 , wherein the operations further comprise:
 mapping each of the one or more data included in the data subset to a data space, and   wherein determining the similarity level comprises determining based on a location in the data space of each of the one or more data included in the data subset.   
     
     
         10 . The non-transitory computer readable medium according to  claim 1 , wherein the data subset comprises a unique pair selected from the one or more data included in the dataset. 
     
     
         11 . The non-transitory computer readable medium according to  claim 1 , wherein the data includes sensor data collected during manufacturing of the article in a manufacturing environment. 
     
     
         12 . A method of processing data executed on one or more processors of a computing device, the method comprising:
 determining an uncertainty level with respect to labeling criteria for each of one or more data included in a dataset;   determining a similarity level for one or more data included in a data subset, wherein one or both of determining the uncertainty level and determining the similarity level are based on determining a location, in a data space, of the one or more data included in the respective one of the dataset and the data subset;   selecting at least some of data included in the dataset based on a calculation that correlates the uncertainty level and the similarity level by adding relative weights on each of the uncertainty level and the similarity level; and   labeling the selected dated.   
     
     
         13 . A computing device for processing data, comprising:
 one or more processors; and   a memory for storing computer programs executable on the one or more processors,   wherein the one or more processors are configured to:
 determine an uncertainty level with respect to labeling criteria for each of one or more data included in a dataset; 
 determine a similarity level for one or more data included in a data subset, wherein one or both of determining the uncertainty level and determining the similarity level are based on determining a location, in a data space, of the one or more data included in the respective one of the dataset and the data subset; 
 select at least some of data included in the dataset based on a calculation that correlates the uncertainty level and the similarity level by adding relative weights on each of the uncertainty level and the similarity level; and 
 label the selected data.