Learning system, learning device, learning method, learning program, teacher data creation device, teacher data creation method, teacher data creation program, terminal device, and threshold value changing device

Abstract

A training system comprises a training device, and a training data creation device for the training device. The training device trains a neural network by means of a backpropagation algorithm. The training data creation device acquires any one of a positive evaluation indicating that content of the input data coincides with the label, a negative evaluation indicating that content of the input data does not coincide with the label, and an ignorable evaluation indicating exclusion from a training target label, for each label regarding the input data to create training data. In training the neural network for training, the training system adjusts the weight coefficient for the intermediate layer such that the recognition score of the label with the positive evaluation or the negative evaluation comes closer to the ground-truth score of the positive evaluation or the negative evaluation, and makes the recognition score of the label with the ignorable evaluation not affect the adjustment of the weight coefficient for the intermediate layer.

1 . A threshold value changing device changing a threshold value in a terminal device, the terminal device acquiring recognition target data, outputting by a neural network a recognition score indicating a degree of coincidence between content of the recognition target data and a predetermined label, and outputting a recognition result indicating whether or not the content of the recognition target data coincides with the predetermined label by use of a threshold value set in advance for the recognition score and the recognition score, the threshold value changing device comprising:
 an evaluation data acquisition unit configured to acquire evaluation data including input data and a truth evaluation of the predetermined label, the truth evaluation being associated with the input data, and indicating whether content of the input data coincides with the predetermined label to have a positive evaluation, or the content of the input data does not coincide with the predetermined label to have a negative evaluation,   a terminal data acquisition unit configured to acquire a ratio between the positive evaluation and a negative evaluation of data associated with the terminal device,   a recognition score acquisition unit configured to acquire the recognition score of the predetermined label for the input data from the neural network or a neural network having a weight coefficient the same as a weight coefficient for the former neural network,   a computation unit configured to compute the number of pieces of data recognized as the positive evaluation for the input data with the truth evaluation being the positive evaluation and the number of pieces of data recognized as the positive evaluation for the input data with the truth evaluation being the negative evaluation by use of the recognition score of the predetermined label acquired by the recognition score acquisition unit and the threshold value to compute a precision for the predetermined label using the computed number of pieces of data, and   a changing unit configured to change the threshold value by use of the precision computed by the computation unit, wherein   the computation unit corrects the number of pieces of data recognized as the positive evaluation for the input data with the truth evaluation being the negative evaluation by use of a ratio between the positive evaluation and the negative evaluation of the evaluation data, and the ratio between the positive evaluation and the negative evaluation of the data associated with the terminal device to compute the precision using the corrected number of pieces of data.   
     
     
         2 . The threshold value changing device according to  claim 1 , wherein
 the computation unit computes a recall and the precision for the predetermined label, and   the changing unit changes the threshold value to a recognition score with a harmonic mean between the recall and the precision being maximum.   
     
     
         3 . The threshold value changing device according to  claim 1  wherein
 the terminal data acquisition unit acquires the ratio between the positive evaluation and the negative evaluation of the data associated with the terminal device, based on the recognition result by the neural network of the terminal device, or a result of annotation by a user of the terminal device. 
 
     
     
         4 . The threshold value changing device according to  claim 1 , wherein
 the terminal data acquisition unit acquires the ratio between the positive evaluation and the negative evaluation of the data associated with the terminal device, based on an operation on the terminal device by a user or terminal information.