Efficient machine learning model architecture selection

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing a machine learning model that is trained to perform a machine learning task. In one aspect, a method comprises receiving a request to train a machine learning model on a set of training examples; determining a set of one or more meta-data values characterizing the set of training examples; using a mapping function to map the set of meta-data values characterizing the set of training examples to data identifying a particular machine learning model architecture; selecting, using the particular machine learning model architecture, a final machine learning model architecture for performing the machine learning task; and training a machine learning model having the final machine learning model architecture on the set of training examples.

What is claimed is: 
     
         1 . A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations comprising:
 receiving a request to train a machine learning model to perform a machine learning task on a set of training examples;   obtaining a plurality of machine learning model architectures each capable of performing the machine learning task;   for each respective machine learning model architecture of the plurality of machine learning model architectures:
 training a corresponding machine learning model comprising the respective machine learning model architecture; and 
 determining a corresponding prediction performance for the machine learning task using the corresponding machine learning model comprising the respective machine learning model architecture; 
   selecting a respective one of the machine learning model architectures from among the plurality of machine learning model architectures that comprises a greatest corresponding prediction performance for the machine learning task; and   further training the machine learning model comprising the respective one of the machine learning model architectures using the set of training examples.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the operations further comprise generating one or more new sets of training examples from the set of training examples. 
     
     
         3 . The computer-implemented method of  claim 2 , wherein generating the one or more new sets of training examples from the set of training examples comprises modifying the set of training examples. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein the plurality of machine learning model architectures comprises at least one of:
 a neural network model;   a random forest model;   a support vector machine model; or   a linear regression model.   
     
     
         5 . The computer-implemented method of  claim 4 , wherein the neural network model comprises neural network layers. 
     
     
         6 . The computer-implemented method of  claim 5 , wherein the neural network layers comprise at least one of:
 fully-connected layers;   convolutional layers;   recurrent layers; or   batch-normalization layers.   
     
     
         7 . The computer-implemented method of  claim 1 , wherein the machine learning task comprises processing an image to predict a category associated with the processed image. 
     
     
         8 . The computer-implemented method of  claim 1 , wherein the machine learning task comprises processing input text in a first language to predict output text in a second language. 
     
     
         9 . The computer-implemented method of  claim 1 , wherein the machine learning task comprises processing a spoken utterance to predict output text that transcribes the spoken utterance. 
     
     
         10 . The computer-implemented method of  claim 1 , wherein the operations further comprise determining one or more meta-data values characterizing the set of training examples. 
     
     
         11 . A system comprising:
 data processing hardware; and   memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
 receiving a request to train a machine learning model to perform a machine learning task on a set of training examples; 
 obtaining a plurality of machine learning model architectures each capable of performing the machine learning task; 
 for each respective machine learning model architecture of the plurality of machine learning model architectures:
 training a corresponding machine learning model comprising the respective machine learning model architecture; and 
 determining a corresponding prediction performance for the machine learning task using the corresponding machine learning model comprising the respective machine learning model architecture; 
 
 selecting a respective one of the machine learning model architectures from among the plurality of machine learning model architectures that comprises a greatest corresponding prediction performance for the machine learning task; and 
 further training the machine learning model comprising the respective one of the machine learning model architectures using the set of training examples. 
   
     
     
         12 . The system of  claim 11 , wherein the operations further comprise generating one or more new sets of training examples from the set of training examples. 
     
     
         13 . The system of  claim 12 , wherein generating the one or more new sets of training examples from the set of training examples comprises modifying the set of training examples. 
     
     
         14 . The system of  claim 11 , wherein the plurality of machine learning model architectures comprises at least one of:
 a neural network model;   a random forest model;   a support vector machine model; or   a linear regression model.   
     
     
         15 . The system of  claim 14 , wherein the neural network model comprises neural network layers. 
     
     
         16 . The system of  claim 15 , wherein the neural network layers comprise at least one of:
 fully-connected layers;   convolutional layers;   recurrent layers; or   batch-normalization layers.   
     
     
         17 . The system of  claim 11 , wherein the machine learning task comprises processing an image to predict a category associated with the processed image. 
     
     
         18 . The system of  claim 11 , wherein the machine learning task comprises processing input text in a first language to predict output text in a second language. 
     
     
         19 . The system of  claim 11 , wherein the machine learning task comprises processing a spoken utterance to predict output text that transcribes the spoken utterance. 
     
     
         20 . The system of  claim 11 , wherein the operations further comprise determining one or more meta-data values characterizing the set of training examples.