Model ensemble generation

Abstract

A method of generating a model ensemble may be provided. A method may include training a base model including a plurality of layers. The method may also include generating a plurality of models for the neural network based on the base model. Each model of the plurality of models includes a plurality of layers. Further, the method may include modifying a layer of each of the plurality of models such that each model of the plurality of models includes a layer modified with respect to an associated layer of each of the base model and each of the other plurality of models. In addition, the method may include tuning each modified layer of the plurality of models.

What is claimed is: 
     
         1 . A method of generating a model ensemble, comprising:
 training, via at least one processor, a base model including a plurality of layers;   generating, via the at least one processor, a plurality of models for the model ensemble based on the base model, each model of the plurality of models including a plurality of layers;   modifying, via the at least one processor, a layer of each of the plurality of models such that each model of the plurality of models includes a layer modified with respect to an associated layer of each of the base model and an associated layer of each of the other plurality of models; and   tuning, via the at least one processor, each modified layer of the plurality of models.   
     
     
         2 . The method of  claim 1 , further comprising:
 receiving an output from each of the plurality of models; and   generating, via the at least one processor, a model ensemble output based on the output of each of the plurality of models.   
     
     
         3 . The method of  claim 1 , wherein modifying comprises modifying the layer of each of the plurality of models based on at least one of clustering and quantization. 
     
     
         4 . The method of  claim 1 , wherein modifying comprises modifying at least one training parameter of the layer of each of the plurality of models. 
     
     
         5 . The method of  claim 4 , wherein modifying at least one training parameter of the layer comprises modifying at least one of a number of bits of the layer, a number of neurons of the layer, weights for one or more connections of the layer, and a number of connections of the layer. 
     
     
         6 . The method of  claim 1 , wherein generating comprises generating, via the at least one processor, each of the plurality of models as a replica of the base model. 
     
     
         7 . The method of  claim 1 , wherein tuning each modified layer comprises tuning each modified layer with an X number of epochs. 
     
     
         8 . The method of  claim 7 , wherein training the base model comprises training the base layer with  10 X number of epochs. 
     
     
         9 . The method of  claim 1 , further comprising:
 arbitrarily selecting at least one additional layer in at least one model for modification;   modifying the selected at least one additional layer; and   tuning the selected at least one additional layer.   
     
     
         10 . The method of  claim 1 , wherein training the base model comprises training the base model via random initialization. 
     
     
         11 . One or more non-transitory computer-readable media that include instructions that, when executed by one or more processors, are configured to cause the one or more processors to perform operations, the operations comprising:
 training a base model including a plurality of layers;   generating a plurality of models for a model ensemble based on the base model, each model of the plurality of models including a plurality of layers;   modifying a layer of each of the plurality of models such that each model of the plurality of models includes a layer modified with respect to an associated layer of each of the base model and an associated layer of each of the other plurality of models; and   tuning each modified layer of the plurality of models.   
     
     
         12 . The computer-readable media of  claim 11 , the operations further comprising:
 receiving an output from each of the plurality of models; and   generating a model ensemble output based on the output of each of the plurality of models.   
     
     
         13 . The computer-readable media of  claim 11 , wherein modifying comprises modifying the layer of each of the plurality of models based on at least one of clustering and quantization. 
     
     
         14 . The computer-readable media of  claim 11 , wherein modifying comprises modifying at least one training parameter of the layer of each of the plurality of models. 
     
     
         15 . The computer-readable media of  claim 14 , wherein modifying at least one training parameter of the layer comprises modifying at least one of a number of bits of the layer, a number of neurons of the layer, weights for one or more connections of the layer, and a number of connections of the layer. 
     
     
         16 . The computer-readable media of  claim 11 , wherein generating comprises generating, via the at least one processor, each of the plurality of models as a replica of the base model. 
     
     
         17 . The computer-readable media of  claim 11 , wherein tuning each modified layer comprises tuning each modified layer with an X number of epochs. 
     
     
         18 . The computer-readable media of  claim 17 , wherein training the base model comprises training the base layer with  10 X number of epochs. 
     
     
         19 . The computer-readable media of  claim 11 , the operations further comprising:
 arbitrarily selecting at least one additional layer in at least one model for modification;   modifying the selected at least one additional layer; and   tuning the selected at least one additional layer.   
     
     
         20 . The computer-readable media of  claim 11 , wherein training the base model comprises training the base model via random initialization.