Federated representation learning with consistency regularization

Abstract

The present invention relates to the technical field of federated learning. Subject matter of the present invention is a method for (re-)training a federated learning system, a computer system for carrying out the method, and a non-transitory computer-readable storage medium comprising processor-executable instructions with which to perform an operation for (re-)training a federated learning system.

1 . A computer system comprising a plurality of edge devices, wherein:
 each edge device has access to a shared global model, wherein the shared global model is configured to:
 generate a feature vector based at least partially on input data provided by the edge device and on global model parameters; 
   each edge device comprises a task performing model which is configured to:
 receive the feature vector generated by the shared global model based on the input data provided by the edge device, and 
 perform a task based at least partially on the feature vector and on task performing model parameters; 
   the computer system is configured to perform a training or a re-training, the training or re-training comprising:
 receiving a new set of training data for a first edge device, the first edge device comprising a first task performing model which is configured to perform a first task based at least partially on first task performing model parameters, 
 training the shared global model and the first task performing model using the training data, the training comprising modifying the global model parameters and the first task performing model parameters so that a loss value calculated from a loss function is minimized, wherein the loss function is configured to:
 reward modifications of the first task performing model parameters and the global model parameters which lead to an improved performance of the first task; 
 reward modifications of the global model parameters which lead to an improvement of the shared global model; and 
 penalize modifications of the global model parameters which lead to a worse performance of tasks performed by other task performing models of other edge devices of the plurality of edge devices. 
 
   
     
     
         2 . The computer system of  claim 1 , wherein the computer system comprises a central server, wherein a copy of the shared global model is stored on the central server and on each edge device, wherein the central server is configured to update the shared global model on each edge device if an updated version is available. 
     
     
         3 . The computer system of  claim 1 , wherein there are at least two edge devices which comprise different task performing models to perform different tasks. 
     
     
         4 . The computer system of  claim 1 , wherein the tasks performed by the task performing models are selected from: classification task, regression task, data generation task, image segmentation task, reconstruction task, image quality enhancement task and/or combinations thereof. 
     
     
         5 . The computer system of  claim 1 , wherein each model is or comprises a machine learning model based on an artificial neural network. 
     
     
         6 . The computer system of  claim 1 , wherein the shared global model is or comprises an encoder-decoder architecture, the encoder being configured to generate a feature vector from input data and the decoder being configured to reconstruct input data from a feature vector. 
     
     
         7 . The computer system  claim 1 , wherein a consistency data set is stored on each edge device, wherein each consistency data set comprises consistency input data and consistency target data, wherein the training or re-training comprises:
 computing a first loss value L (1) , the first loss value L (1)  quantifying the impact of modifications of the global model parameters and of the first task performing model parameters on the performance of the first task;   computing a feature generation loss value L GM , the feature generation loss value L GM  quantifying the impact of modifications of the global model parameters on the quality of the feature vector generation, wherein the shared global model is set up as an encoder-decoder, wherein the encoder is configured to receive input data and generate, at least partially on the basis of the global model parameters, a feature vector from the input data, and the decoder is configured to reconstruct, at least partially on the basis of the global model parameters, the input data from the feature vector, wherein the quality of the feature vector generation is quantified by computing a reconstruction loss;   for each edge device E (j)  other than the first edge device:
 inputting the consistency input data into the shared global model, 
 receiving from the shared global model a feature vector, 
 inputting the feature vector into the task performing model stored on the edge device, 
 receiving a result from the task performing model, and 
 computing, at least partially on the basis of the received result and the consistency target data, a consistency loss L (j) , the consistency loss quantifying the impact of modifications of the global model parameters on the deviations between the result and the consistency target; 
   computing a total loss value L from the feature generation loss value L GM , the first loss value L (1)  and all consistency losses L (j) ; and   modifying the global model parameters and the first task performing model parameters so that the total loss value L is minimized.   
     
     
         8 . The computer system of  claim 1 , wherein the input data and the training data are medical data of one or more patients. 
     
     
         9 . The computer system of  claim 1 , wherein one or more tasks performed by one or more task performing models of one or more edge devices comprises the detection of signs indicative of one or more diseases. 
     
     
         10 . The computer system of  claim 1 , wherein the one or more diseases is/are: CTEPH, ARDS and/or COPD. 
     
     
         11 . A computer-implemented method of training or re-training a federated learning system, the method comprising:
 providing a federated learning system comprising at least two edge devices, a first edge device and a second edge device, wherein:
 the first edge device and the second edge device have access to a shared global model, wherein the shared global model is configured to generate a feature vector based at least partially on input data provided by the first edge device or the second edge device and global model parameters, 
 the first edge device comprises a first task performing model, wherein the first task performing model is configured to perform a first task based at least partially on the feature vector generated by the shared global model based on the input data provided by the first edge device and on first task performing model parameters, and 
 the second edge device comprises a second task performing model, wherein the second task performing model is configured to perform a second task based at least partially on the basis of the feature vector generated by the shared global model based on input data provided by the second edge device and on second task performing model parameters; 
   training or re-training of the federated learning system, wherein the training or re-training comprises:
 inputting first input data into the shared global model and receiving a first feature vector, 
 inputting the first feature vector into the first task performing model and receiving a first task result, 
 inputting second input data into the shared global model, and receiving a second feature vector, 
 inputting the second feature vector into the second task performing model and receiving a second task result, 
 calculating a loss value by using a loss function, wherein the loss function is configured to:
 reward modifications of the first task performing model parameters and the global model parameters which lead to an improved performance of the first task, 
 reward modifications of the global model parameters which lead to an improvement of the shared global model, and 
 penalize modifications of the global model parameters which lead to a worse performance of the second task, 
 modifying the first task performing model parameters and the global model parameters, based, at least partially, on minimizing the loss value. 
 
   
     
     
         12 . The method of  claim 11 , wherein the first task performance model is configured to perform a different task than the second task performing model. 
     
     
         13 . The method of  claim 11 , wherein a consistency data set is stored on the second edge device, the consistency data set comprising consistency input data and consistency target data, wherein the training or re-training comprises the following steps:
 receiving new training data for the first edge device, the new training data comprising first input data and first target data,   inputting the first input data into the shared global model, and receiving a first feature vector,   inputting the first feature vector into the first task performing model and receiving a first task result,   calculating a first loss value, the first loss value quantifying the deviations between the first task result and the first target data,   inputting the consistency input data into the shared global model, and receiving a second feature vector,   inputting the second feature vector into the second task performing model and receiving a second task result,   calculating a second loss value, the second loss value quantifying the deviations between the second task result and the consistency target data,   reconstructing the first input data from the first feature vector using the shared global model,   calculating a third loss value, the third loss value quantifying the deviations between the first input data and the reconstructed first input data,   calculating a total loss value on the basis of the first loss value, the second loss value and the third loss value, and   modifying the first task performing model parameters and the global model parameters, based, at least partially, on minimizing the total loss value.   
     
     
         14 . A non-transitory computer-readable storage medium comprising processor-executable instructions with which to perform an operation for training or re-training a federated learning system, the federated learning system comprising at least two edge devices, a first edge device and a second edge device, wherein:
 the first edge device and the second edge device have access to a shared global model, wherein the shared global model is configured to generate a feature vector based at least partially on input data provided by the first edge device or by the second edge device and on global model parameters,   the first edge device comprises a first task performing model, wherein the first task performing model is configured to perform a first task based at least partially on the feature vector generated by the shared global model on the basis of the based on input data provided by the first edge device and on first task performing model parameters, and   the second edge device comprises a second task performing model, wherein the second task performing model is configured to perform a second task based at least partially on the basis of the feature vector generated by the shared global model based on the input data provided by the second edge device and on second task performing model parameters,   
       wherein the operation comprises:
 inputting first input data into the shared global model, and receiving a first feature vector, 
 inputting the first feature vector into the first task performing model and receiving a first task result, 
 inputting second input data into the shared global model, and receiving a second feature vector, 
 inputting the second feature vector into the second task performing model and receiving a second task result, 
 calculating a loss value by using a loss function, wherein the loss function is configured to:
 reward modifications of the first task performing model parameters and the global model parameters which lead to an improved performance of the first task, and 
 reward modifications of the global model parameters which lead to an improvement of the shared global model, and 
 penalize modifications of the global model parameters which lead to a worse performance of the second task, 
 
 modifying the first task performing model parameters and the global model parameters, based, at least partially, on minimizing the loss value. 
 
     
     
         15 . The computer system of  claim 1 , wherein the computer system is configured for medical use, in particular for performing tasks on medical data of patients.