Systems and Methods for Deploying and Updating Neural Networks at the Edge of a Network

Abstract

Methods, devices and system for updating a neural network on an edge device that has low-bandwidth uplink capability include a centralized site/device that is configured to train and send the neural network to the edge device. In response, the centralized site/device may receive neural network information from the edge device that includes all or portions of a dataset, output activations, and/or overall inference result that is collected or generated in the edge device. The centralized site/device may use the received neural network information to update all or a part of the trained neural network, generate updated neural network information based on the updated neural network, and send the updated neural network information to the edge device.

What is claimed is: 
     
         1 . A method of updating a neural network on an edge device that has low-bandwidth uplink capability, comprising:
 training, by a processor in a centralized site/device, the neural network;   sending, by the processor, the trained neural network to the edge device;   receiving, by the processor, neural network information from the edge device, the received neural network information including at least a portion of at least one or more of a dataset, an activation, or an overall inference result collected or generated in the edge device;   using, by the processor, the received neural network information to update all or a part of the trained neural network;   generating, by the processor, updated neural network information based on the updated neural network; and   sending, by the processor, the updated neural network information to the edge device.   
     
     
         2 . The method of  claim 1 , wherein sending the trained neural network to the edge device comprises sending the trained neural network to an edge device that has been deployed. 
     
     
         3 . The method of  claim 1 , wherein using the received neural network information to update all or a part of the trained neural network and generating the updated neural network information based on the updated neural network comprises:
 generating a neural network difference model by comparing the updated neural network to the trained neural network.   
     
     
         4 . The method of  claim 3 , wherein generating the neural network difference model by comparing the updated neural network to the trained neural network comprises:
 generating a patch that identifies the differences between the updated neural network and the trained neural network via one of:
 layer freezing using a minimum size technique; 
 layer freezing using a minimum delta technique; 
 weights freezing using the minimum size technique; or 
 weights freezing using the minimum delta technique. 
   
     
     
         5 . The method of  claim 3 , wherein generating the neural network difference model by comparing the updated neural network to the trained neural network comprises:
 determining one or more neural network layers or one or more neural network weights of the one or more neural network layers to freeze based on a mean of activations of layers in the neural network.   
     
     
         6 . The method of  claim 1 , further comprising:
 receiving, by the edge device, the trained neural network;   collecting, by the edge device, the dataset from sensors of the edge device;   applying, by the edge device, the collected dataset as inputs to the received neural network to generate activations and the overall inference result;   storing, by the edge device, at least a portion of at least one or more of the collected dataset, the generated activations or the overall inference result in a memory of the edge device; and   sending, by the edge device, the neural network information that includes at least a portion of at least one or more of the collected dataset, the generated activations or the overall inference result to the centralized site/device.   
     
     
         7 . The method of  claim 6 , further comprising:
 receiving, by the edge device, the updated neural network information;   generating, by the edge device, an updated neural network based on the received trained neural network and the received updated neural network information; and   applying a second dataset as input to the updated neural network to generate second inference results.   
     
     
         8 . The method of  claim 7 , wherein receiving the updated neural network information comprises receiving a neural network difference model. 
     
     
         9 . The method of  claim 1 , wherein:
 training the neural network comprises:
 collecting training data from one or more of a plurality of edge devices; 
 labelling the collected training data; 
 selecting two or more lightweight neural networks; 
 generating an ensemble based on the selected neural networks; and 
 using the labelled training data to train the ensemble; and 
   sending the trained neural network to the edge device comprises:
 sending the trained ensemble and an ensemble aggregation function to the edge device. 
   
     
     
         10 . The method of  claim 9 , wherein using the received neural network information to update all or a part of the trained neural network comprises:
 adding a neural network to the trained ensemble.   
     
     
         11 . The method of  claim 9 , wherein using the received neural network information to update all or a part of the trained neural network comprises:
 updating the ensemble aggregation function based on a result of analyzing the received neural network information; and   updating all or a part of the trained neural network based on the updated ensemble aggregation function.   
     
     
         12 . The method of  claim 6 , wherein:
 receiving the trained neural network comprises receiving a trained ensemble; and   applying the collected dataset as inputs to the received neural network to generate the activations and the overall inference result comprises applying the collected dataset as inputs to the received ensemble to generate the activations and the overall inference result.   
     
     
         13 . The method of  claim 1 , wherein training the neural network comprises:
 generating a stratified neural network that includes large data volume parts and small data parts.   
     
     
         14 . The method of  claim 13 , wherein sending the updated neural network information to the edge device comprises sending the small data parts of the stratified neural network to the edge device. 
     
     
         15 . The method of  claim 13 , wherein generating the stratified neural network that includes the large data volume parts and the small data parts comprises:
 generating the stratified neural network to include:
 a large data volume part that include a feature identification layer; and 
 a small data part that includes a fully connected layer. 
   
     
     
         16 . The method of  claim 13 , wherein generating the stratified neural network that includes the large data volume parts and the small data parts comprises:
 generating the stratified neural network to include:
 large data volume parts that include multiple partial layers that are not cross-connected; and 
 small data parts that include cross-connected weights between the multiple partial layers in the large data volume parts. 
   
     
     
         17 . The method of  claim 13 , wherein generating the stratified neural network that includes the large data volume parts and the small data parts comprises:
 generating the stratified neural network to include:
 large data volume parts that include layers with a higher numerical precision; and 
 small data parts that include layers with a lower numerical precision. 
   
     
     
         18 . The method of  claim 13 , wherein using the received neural network information to update all or a part of the trained neural network and generating the updated neural network information based on the updated neural network comprises:
 retraining only the small data parts of the stratified neural network.   
     
     
         19 . A centralized site/device, comprising
 a processor is configured with processor-executable instructions to perform operations comprising:
 training a neural network; 
 sending the trained neural network to an edge device that has low-bandwidth uplink capability; 
 receiving neural network information from the edge device, the received neural network information including at least a portion of at least one or more of a dataset, an activation, or an overall inference result collected or generated in the edge device; 
 using the received neural network information to update all or a part of the trained neural network; 
 generating updated neural network information based on the updated neural network; and 
 sending the updated neural network information to the edge device. 
   
     
     
         20 . A non-transitory computer readable storage medium having stored thereon processor-executable software instructions configured to cause a processor in a centralized site/device to perform operations for updating a neural network on an edge device that has low-bandwidth uplink capability, the operations comprising:
 training a neural network;   sending the trained neural network to an edge device that has low-bandwidth uplink capability;   receiving neural network information from the edge device, the received neural network information including at least a portion of at least one or more of a dataset, an activation, or an overall inference result collected or generated in the edge device;   using the received neural network information to update all or a part of the trained neural network;   generating updated neural network information based on the updated neural network; and   sending the updated neural network information to the edge device.