Multi-task neural networks with task-specific paths

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using multi-task neural networks. One of the methods includes receiving a first network input and data identifying a first machine learning task to be performed on the first network input; selecting a path through the plurality of layers in a super neural network that is specific to the first machine learning task, the path specifying, for each of the layers, a proper subset of the modular neural networks in the layer that are designated as active when performing the first machine learning task; and causing the super neural network to process the first network input using (i) for each layer, the modular neural networks in the layer that are designated as active by the selected path and (ii) the set of one or more output layers corresponding to the identified first machine learning task.

What is claimed is: 
     
         1 . A method of training a super neural network on a plurality of machine learning tasks, the plurality of machine learning tasks comprising a first machine learning task and a last machine learning task, the super neural network comprising:
 a plurality of layers arranged in a stack one after the other, wherein each layer comprises a respective plurality of modular neural networks, and wherein the modular neural networks in each layer are each configured to receive a layer input for the layer and to generate a respective modular output; and   a plurality of sets of one or more output layers, wherein each set of output layers corresponds to a different machine learning task from the plurality of machine learning tasks, and wherein each set of one or more output layers is configured to receive a stack output and to generate a neural network output that is specific to the corresponding machine learning task, and the method comprising:
 obtaining first training data for the first machine learning task; and 
 training the super neural network on the first training data to determine a best fit path through the plurality of layers for the first machine learning task while (i) holding fixed values of parameters of any modular neural networks that are in best fit paths for any of the plurality of machine learning tasks that the super neural network has already been trained on prior to the first machine learning task and (ii) adjusting values of parameters of other modular neural networks that are not in best fit paths for any of the machine learning tasks that the super neural network has already been trained on. 
   
     
     
         2 . The method of  claim 1 , further comprising:
 re-initializing values of parameters of modular neural networks that are not in the best fit path for the first machine learning task or any of the best fit paths for any of the machine learning tasks that the super neural network has already been trained on;   obtaining second training data for a second machine learning task; and   training the super neural network on the second training data to determine a best fit path through the plurality of layers for the second machine learning task while (i) holding fixed values of parameters of any modular neural networks that are in the best fit path for the first machine learning task or any of the best fit paths for any of the machine learning tasks that that the super neural network has already been trained on and (ii) adjusting values of parameters of other modular neural networks that are not in the best fit path for the first machine learning task or any of the best fit paths for any of the machine learning tasks that the super neural network has already been trained on.   
     
     
         3 . The method of  claim 1 , wherein training the super neural network on the first training data to determine a best fit path through the plurality of layers for the first machine learning task comprises:
 selecting a plurality of candidate paths;   for each of the plurality of candidate paths:
 evaluating a fitness of the candidate path by training the super neural network on the first machine learning task with the modular neural networks in the candidate path designated as active; and 
   selecting a candidate path with the best fitness as a current best path.   
     
     
         4 . The method of  claim 3 , further comprising:
 mutating the current best path to generate a mutated current best path; and   including the mutated current best path in a new plurality of candidate paths for evaluation.   
     
     
         5 . The method of  claim 4 , wherein mutating the current best path comprises:
 for each modular neural network designated as active by the current best path:   determining whether to replace the modular neural network; and in response to determining to replace the modular neural network, designating the modular neural network as inactive and designating another modular neural network in the same layer as active.   
     
     
         6 . The method of  claim 5 , wherein determining whether to replace the modular neural network comprises determining to replace the modular neural network with probability that is equal to or proportional to 1 divided by a total number of modular neural networks in the current best path. 
     
     
         7 . The method of  claim 3 , wherein training the super neural network on the first machine learning task with the modular neural networks in the candidate path designated as active comprises training the super neural network on an initial plurality of candidate paths in parallel. 
     
     
         8 . The method of  claim 7 , wherein selecting the plurality of candidate paths comprises, for each candidate path in the initial plurality:
 determining whether any of a subset of other candidate paths in the initial plurality has a better fitness than the candidate path; and if any of the subset have a better fitness:   identifying the candidate path in the subset that has a best fitness of the other candidate paths in the subset;   mutating the identified candidate path by changing one or more of the active modular neural networks in the identified candidate path; and   replacing the candidate path with the mutated identified candidate path.   
     
     
         9 . The method of  claim 8 , wherein selecting the plurality of candidate paths further comprises, for each candidate path in the initial plurality: if none of the candidate paths in the subset have a better fitness, maintaining the candidate path for further training. 
     
     
         10 . A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for training a super neural network on a plurality of machine learning tasks, the plurality of machine learning tasks comprising a first machine learning task and a last machine learning task, the super neural network comprising:
 a plurality of layers arranged in a stack one after the other, wherein each layer comprises a respective plurality of modular neural networks, and wherein the modular neural networks in each layer are each configured to receive a layer input for the layer and to generate a respective modular output; and   a plurality of sets of one or more output layers, wherein each set of output layers corresponds to a different machine learning task from the plurality of machine learning tasks, and wherein each set of one or more output layers is configured to receive a stack output and to generate a neural network output that is specific to the corresponding machine learning task, and the operations comprising:
 obtaining first training data for the first machine learning task; and 
 training the super neural network on the first training data to determine a best fit path through the plurality of layers for the first machine learning task while (i) holding fixed values of parameters of any modular neural networks that are in best fit paths for any of the plurality of machine learning tasks that the super neural network has already been trained on prior to the first machine learning task and (ii) adjusting values of parameters of other modular neural networks that are not in best fit paths for any of the machine learning tasks that the super neural network has already been trained on. 
   
     
     
         11 . The system of  claim 10 , the operations further comprising:
 re-initializing values of parameters of modular neural networks that are not in the best fit path for the first machine learning task or any of the best fit paths for any of the machine learning tasks that the super neural network has already been trained on;   obtaining second training data for a second machine learning task; and   training the super neural network on the second training data to determine a best fit path through the plurality of layers for the second machine learning task while (i) holding fixed values of parameters of any modular neural networks that are in the best fit path for the first machine learning task or any of the best fit paths for any of the machine learning tasks that that the super neural network has already been trained on and (ii) adjusting values of parameters of other modular neural networks that are not in the best fit path for the first machine learning task or any of the best fit paths for any of the machine learning tasks that the super neural network has already been trained on.   
     
     
         12 . The system of  claim 10 , wherein training the super neural network on the first training data to determine a best fit path through the plurality of layers for the first machine learning task comprises:
 selecting a plurality of candidate paths;   for each of the plurality of candidate paths:
 evaluating a fitness of the candidate path by training the super neural network on the first machine learning task with the modular neural networks in the candidate path designated as active; and 
   selecting a candidate path with the best fitness as a current best path.   
     
     
         13 . The system of  claim 12 , the operations further comprising:
 mutating the current best path to generate a mutated current best path; and   including the mutated current best path in a new plurality of candidate paths for evaluation.   
     
     
         14 . The system of  claim 13 , wherein mutating the current best path comprises:
 for each modular neural network designated as active by the current best path: determining whether to replace the modular neural network; and in response to determining to replace the modular neural network, designating the modular neural network as inactive and designating another modular neural network in the same layer as active.   
     
     
         15 . The system of  claim 14 , wherein determining whether to replace the modular neural network comprises determining to replace the modular neural network with probability that is equal to or proportional to 1 divided by a total number of modular neural networks in the current best path. 
     
     
         16 . The system of  claim 12 , wherein training the super neural network on the first machine learning task with the modular neural networks in the candidate path designated as active comprises training the super neural network on an initial plurality of candidate paths in parallel. 
     
     
         17 . The system of  claim 16 , wherein selecting the plurality of candidate paths comprises, for each candidate path in the initial plurality:
 determining whether any of a subset of other candidate paths in the initial plurality has a better fitness than the candidate path; and if any of the subset have a better fitness:   identifying the candidate path in the subset that has a best fitness of the other candidate paths in the subset;   mutating the identified candidate path by changing one or more of the active modular neural networks in the identified candidate path; and   replacing the candidate path with the mutated identified candidate path.   
     
     
         18 . The system of  claim 17 , wherein selecting the plurality of candidate paths further comprises, for each candidate path in the initial plurality: if none of the candidate paths in the subset have a better fitness, maintaining the candidate path for further training. 
     
     
         19 . One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training a super neural network on a plurality of machine learning tasks, the plurality of machine learning tasks comprising a first machine learning task and a last machine learning task, the super neural network comprising:
 a plurality of layers arranged in a stack one after the other, wherein each layer comprises a respective plurality of modular neural networks, and wherein the modular neural networks in each layer are each configured to receive a layer input for the layer and to generate a respective modular output; and   a plurality of sets of one or more output layers, wherein each set of output layers corresponds to a different machine learning task from the plurality of machine learning tasks, and wherein each set of one or more output layers is configured to receive a stack output and to generate a neural network output that is specific to the corresponding machine learning task, and the operations comprising:
 obtaining first training data for the first machine learning task; and 
 training the super neural network on the first training data to determine a best fit path through the plurality of layers for the first machine learning task while (i) holding fixed values of parameters of any modular neural networks that are in best fit paths for any of the plurality of machine learning tasks that the super neural network has already been trained on prior to the first machine learning task and (ii) adjusting values of parameters of other modular neural networks that are not in best fit paths for any of the machine learning tasks that the super neural network has already been trained on. 
   
     
     
         20 . The non-transitory computer-readable storage media of  claim 19 , the operations further comprising:
 re-initializing values of parameters of modular neural networks that are not in the best fit path for the first machine learning task or any of the best fit paths for any of the machine learning tasks that the super neural network has already been trained on;   obtaining second training data for a second machine learning task; and   training the super neural network on the second training data to determine a best fit path through the plurality of layers for the second machine learning task while (i) holding fixed values of parameters of any modular neural networks that are in the best fit path for the first machine learning task or any of the best fit paths for any of the machine learning tasks that that the super neural network has already been trained on and (ii) adjusting values of parameters of other modular neural networks that are not in the best fit path for the first machine learning task or any of the best fit paths for any of the machine learning tasks that the super neural network has already been trained on.