Distributed machine learning for improved privacy

Abstract

Methods, computer readable media, devices, and systems provide for distributed machine learning. In one aspect, a method of training a model is disclosed. The method includes receiving, by a client device, from one or more servers, an intermediate model, training, by the client device, the intermediate model based on private data, and transmitting, by the client device, to the one or more servers, the trained intermediate model.

What is claimed is: 
     
       1. A method comprising:
 receiving, by a device from a server, an intermediate model; 
 accessing, by the device, training data based on a first message received from the server that indicates a list of active data identifiers, the training data comprising data that is private to the device; 
 storing a mapping between a plurality of models and corresponding lists of one or more input parameters used for training each of the plurality of models; 
 receiving, from the server, model data representing a state of the intermediate model; 
 training, by the device, the intermediate model based on the accessed training data and based on a determination that the device comprises a type of data associated with the intermediate model; and 
 transmitting, by the device, the trained intermediate model to the server without sharing the data that is private to the device. 
 
     
     
       2. The method of  claim 1 , the intermediate model being trained without training a second model, further comprising deleting the training data after training the intermediate model. 
     
     
       3. The method of  claim 1 , further comprising adding random noise to the trained intermediate model prior to transmitting the trained intermediate model to the server. 
     
     
       4. The method of  claim 1 , wherein the server integrates the trained intermediate model received from the device with a second model. 
     
     
       5. The method of  claim 1 , wherein the server integrates the trained intermediate model received from the device with other model data. 
     
     
       6. The method of  claim 5 , wherein the other model data is received by the server from a second device, the second device generating the other model data using additional data that is private to the second device. 
     
     
       7. The method of  claim 1 , further comprising:
 transmitting, to the server, a second message indicating data identifiers of the training data stored on the device, the server selecting the intermediate model based on the data identifiers indicated in the second message. 
 
     
     
       8. The method of  claim 1 , further comprising:
 transmitting, to the server, a second message indicating a model to fit using the training data that has been collected by the device, the server selecting the intermediate model based on the model indicated in the second message. 
 
     
     
       9. The method of  claim 1 , further comprising:
 determining of a list of data identifiers for the intermediate model based on the mapping. 
 
     
     
       10. The method of  claim 1 , further comprising:
 receiving, from the server, data identifiers identifying data for training the intermediate model, wherein training the intermediate model comprises searching a data store to identify data based on the data identifiers, and providing the identified data to the intermediate model to train the intermediate model. 
 
     
     
       11. The method of  claim 10 , wherein the identified data comprises personally identifiable information (PII), and wherein the PII is not present in the trained intermediate model. 
     
     
       12. The method of  claim 1 , where the intermediate model is trained by performing a regular linear regression, logistic regression, neural network training, or Bayesian model training, based on the accessed training data. 
     
     
       13. A system comprising:
 a memory storing instructions; and 
 one or more processors configured to execute the instructions stored in the memory for performing operations comprising: 
 receiving, by a device from a server, an intermediate model; 
 accessing, by the device, training data based on a first message received from the server that indicates a list of active data identifiers, the training data comprising data that is private to the device; 
 storing a mapping between a plurality of models and corresponding lists of one or more input parameters used for training each of the plurality of models; 
 receiving, from the server, model data representing a state of the intermediate model; 
 training, by the device, the intermediate model based on the accessed training data and based on a determination that the device comprises a type of data associated with the intermediate model; and 
 transmitting, by the device, the trained intermediate model to the server without sharing the data that is private to the device. 
 
     
     
       14. The system of  claim 13 , wherein the intermediate model is trained without training a second model, the operations further comprising deleting the training data after training the intermediate model. 
     
     
       15. The system of  claim 13 , the operations further comprising adding random noise to the trained intermediate model prior to transmitting the trained intermediate model to the server. 
     
     
       16. The system of  claim 13 , wherein the server integrates the trained intermediate model received from the device with a second model. 
     
     
       17. The system of  claim 13 , wherein the server integrates the trained intermediate model received from the device with other model data. 
     
     
       18. The system of  claim 17 , wherein the other model data is received by the server from a second device, the second device generating the other model data using additional data that is private to the second device. 
     
     
       19. The system of  claim 13 , the operations further comprising:
 transmitting, to the server, a second message indicating data identifiers of the training data stored on the device, the server selecting the intermediate model based on the data identifiers indicated in the second message. 
 
     
     
       20. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform operations comprising:
 receiving, by a device from a server, an intermediate model; 
 accessing, by the device, training data based on a first message received from the server that indicates a list of active data identifiers, the training data comprising data that is private to the device; 
 storing a mapping between a plurality of models and corresponding lists of one or more input parameters used for training each of the plurality of models; 
 receiving, from the server, model data representing a state of the intermediate model; 
 training, by the device, the intermediate model based on the accessed training data and based on a determination that the device comprises a type of data associated with the intermediate model; and 
 transmitting, by the device, the trained intermediate model to the server without sharing the data that is private to the device.