Implicit bridging of machine learning tasks

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media for performing machine learning tasks. One method includes receiving (i) a model input, and (ii) data identifying a first machine learning task to be performed on the model input to generate a first type of model output for the model input; augmenting the model input with an identifier for the first machine learning task to generate an augmented model input; and processing the augmented model input using a machine learning model, wherein the machine learning model has been trained on training data to perform a plurality of machine learning tasks including the first machine learning task, and wherein the machine learning model has been configured through training to process the augmented model input to generate a machine learning model output of the first type for the model input.

What is claimed is: 
     
         1 - 20 . (canceled) 
     
     
         21 . A computer-implemented method comprising:
 receiving (i) a model input comprising text in a source language, and (ii) data identifying a target language that the text in the source language is to be translated into by the machine learning model;   augmenting the model input with an identifier that identifies at least the target language to generate an augmented model input; and   processing the augmented model input using a machine learning model to generate a model output that is a translation of the model input into the target language, wherein the machine learning model has been trained on training data to translate model inputs into a plurality of different languages including the target language, and wherein the model output is a sequence that includes outputs from a shared vocabulary that includes outputs from all of the plurality of different languages.   
     
     
         22 . The method of  claim 21 , wherein the machine learning model comprises:
 an encoder neural network that is shared between the plurality of different languages and that is configured to receive the augmented model input.   
     
     
         23 . The method of  claim 21 , wherein the machine learning model comprises:
 a decoder neural network that is configured to generate the model output.   
     
     
         24 . The method of  claim 23 , wherein the decoder neural network has an attention mechanism. 
     
     
         25 . The method of  claim 21 , wherein augmenting the model input with an identifier comprises prepending a token identifier that identifies at least the target language to the model input. 
     
     
         26 . The method of  claim 21 , wherein the machine learning model has been trained on training data comprising a plurality of paired datasets, wherein each of the paired datasets comprises an input dataset of text in a respective input language paired with an output dataset in a respective output language. 
     
     
         27 . The method of  claim 26 , wherein the plurality of paired datasets does not include a pairing of datasets comprising an input dataset in the source language paired with an output dataset in the target language. 
     
     
         28 . The method of  claim 21 , wherein the shared vocabulary is a shared word piece vocabulary that includes sub word units that are shared between the plurality of languages. 
     
     
         29 . A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
 receiving (i) a model input comprising text in a source language, and (ii) data identifying a target language that the text in the source language is to be translated into by the machine learning model;   augmenting the model input with an identifier that identifies at least the target language to generate an augmented model input; and   processing the augmented model input using a machine learning model to generate a model output that is a translation of the model input into the target language, wherein the machine learning model has been trained on training data to translate model inputs into a plurality of different languages including the target language, and wherein the model output is a sequence that includes outputs from a shared vocabulary that includes outputs from all of the plurality of different languages.   
     
     
         30 . The system of  claim 29 , wherein the machine learning model comprises:
 an encoder neural network that is shared between the plurality of different languages and that is configured to receive the augmented model input.   
     
     
         31 . The system of  claim 29 , wherein the machine learning model comprises:
 a decoder neural network that is configured to generate the model output.   
     
     
         32 . The system of  claim 31 , wherein the decoder neural network has an attention mechanism. 
     
     
         33 . The system of  claim 29 , wherein augmenting the model input with an identifier comprises prepending a token identifier that identifies at least the target language to the model input. 
     
     
         34 . The system of  claim 29 , wherein the machine learning model has been trained on training data comprising a plurality of paired datasets, wherein each of the paired datasets comprises an input dataset of text in a respective input language paired with an output dataset in a respective output language. 
     
     
         35 . The system of  claim 34 , wherein the plurality of paired datasets does not include a pairing of datasets comprising an input dataset in the source language paired with an output dataset in the target language. 
     
     
         36 . The system of  claim 29 , wherein the shared vocabulary is a shared word piece vocabulary that includes sub word units that are shared between the plurality of languages. 
     
     
         37 . One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
 receiving (i) a model input comprising text in a source language, and (ii) data identifying a target language that the text in the source language is to be translated into by the machine learning model;   augmenting the model input with an identifier that identifies at least the target language to generate an augmented model input; and   processing the augmented model input using a machine learning model to generate a model output that is a translation of the model input into the target language, wherein the machine learning model has been trained on training data to translate model inputs into a plurality of different languages including the target language, and wherein the model output is a sequence that includes outputs from a shared vocabulary that includes outputs from all of the plurality of different languages.   
     
     
         38 . The computer-readable storage media of  claim 37 , wherein augmenting the model input with an identifier comprises prepending a token identifier that identifies at least the target language to the model input. 
     
     
         39 . The computer-readable storage media of  claim 37 , wherein the machine learning model comprises:
 an encoder neural network that is shared between the plurality of different languages and that is configured to receive the augmented model input.   
     
     
         40 . The computer-readable storage media of  claim 37 , wherein the shared vocabulary is a shared word piece vocabulary that includes sub word units that are shared between the plurality of languages.