Training machine learning model

Abstract

Techniques are provided for training machine learning model. According to one aspect, a training data is received by one or more processing units. The machine learning model is trained based on the training data, wherein the training comprises: optimizing the machine learning model based on stochastic gradient descent (SGD) by adding a dynamic noise to a gradient of a model parameter of the machine learning model calculated by the SGD.

What is claimed is: 
     
       1. A method for training a machine learning model, comprising:
 acquiring, by one or more processing units, a training data; and 
 training, by one or more processing units, the machine learning model based on the training data, the training comprising: 
 optimizing, by one or more processing units, the machine learning model based on stochastic gradient descent (SGD), and 
 minimizing privacy leakage by adding a dynamic noise to a gradient of a model parameter of the machine learning model calculated by the SGD. 
 
     
     
       2. The method of  claim 1 , wherein the machine learning model is a convolutional neural networks (CNN) or a recurrent neural network (RNN). 
     
     
       3. The method of  claim 1 , wherein the training data is selected from the group consisting of: pathological data; autopilot data; medical experimental data; biological data; internet of things (IoT) data; social network data; e-commerce data. 
     
     
       4. The method of  claim 1 , wherein the optimizing further comprises minimizing a loss function of the machine learning model. 
     
     
       5. The method of  claim 4 , wherein the added dynamic noise is selected from a predefined noise set. 
     
     
       6. The method of  claim 5 , further comprising assigning a corresponding probability to each of the noises according to the loss function, wherein each of the noises is with a different scale from each other. 
     
     
       7. The method of  claim 6 , wherein the added dynamic noise is selected based on the probability assigned. 
     
     
       8. The method of  claim 5 , wherein the machine learning model is a CNN, and the predefined noise set comprises noises with three different scales and the training data are labeled pathological images. 
     
     
       9. The method of  claim 1 , wherein the noise is a Gaussian noise. 
     
     
       10. A computer system, comprising: a processor;
 a non-transitory computer-readable memory coupled to the processor, the memory comprising instructions that when executed by the processor perform actions of: 
 acquiring, by one or more processing units, a training data; and 
 training, by one or more processing units, the machine learning model based on the training data, the training comprising: 
 optimizing, by one or more processing units, the machine learning model based on stochastic gradient descent (SGD), and 
 minimizing privacy leakage by adding a dynamic noise to a gradient of a model parameter of the machine learning model calculated by the SGD. 
 
     
     
       11. The system of  claim 10 , wherein the machine learning model is a convolutional neural networks (CNN) or a recurrent neural network (RNN). 
     
     
       12. The system of  claim 10 , wherein the training data is selected from the group consisting of: pathological data; autopilot data; medical experimental data; biological data; internet of things (IoT) data; social network data; e-commerce data. 
     
     
       13. The system of  claim 10 , wherein the optimizing further comprises minimizing a loss function of the machine learning model. 
     
     
       14. The system of  claim 13 , wherein the added dynamic noise is selected from a predefined noise set. 
     
     
       15. The system of  claim 14 , further comprising assigning a corresponding probability to each of the noises according to the loss function, wherein each of the noises is with a different scale from each other. 
     
     
       16. The system of  claim 15 , wherein the added dynamic noise is selected based on the probability assigned. 
     
     
       17. The system of  claim 14 , wherein the machine learning model is a CNN, and the predefined noise set comprises noises with three different scales and the training data are labeled pathological images. 
     
     
       18. The system of  claim 10 , wherein the noise is a Gaussian noise. 
     
     
       19. A computer program product for training a machine learning model, comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
 acquiring, by one or more processing units, a training data; 
 training, by one or more processing units, the machine learning model based on the training data, the training comprising:
 optimizing, by one or more processing units, the machine learning model based on stochastic gradient descent (SGD) by adding a dynamic noise, selected from a predefined noise set, to a gradient of a model parameter of the machine learning model calculated by the SGD wherein the optimizing further comprises minimizing a loss function of the machine learning model, and 
 assigning a corresponding probability to each of the noises in the predefined noise set according to the loss function, wherein each of the noises is with a different scale from each other. 
 
 
     
     
       20. The computer program product of  claim 19 , wherein the training data is selected from the group consisting of: pathological data; autopilot data; medical experimental data; biological data; internet of things (IoT) data; social network data; e-commerce data.