Discovering genomes to use in machine learning techniques

Abstract

A facility for identifying combinations of feature and machine learning algorithm parameters, where each combination can be combined with one or more machine learning algorithms to train a model, is disclosed. The facility evaluates each genome based on the ability of a model trained using that genome and a machine learning algorithm to produce accurate results when applied to a validation data set by, for example, generating a fitness or validation score for the trained model and the corresponding genome used to train the model. Genomes that produce fitness scores that exceed a fitness threshold are selected for mutation, mutated, and the process is repeated. These trained models can then be applied to new data to generate predictions for the underlying subject matter.

1 . A system, having a memory and a processor, for discovering machine learning genomes, the system comprising:
 a first component configured to generate a plurality of genomes, wherein each genome identifies at least one feature and at least one parameter for at least one machine learning algorithm, wherein generating a first genome of the plurality of genomes comprises:
 randomly selecting, from among a set of features, one or more of the features, 
 randomly selecting, from among a set of parameters for at least one machine learning algorithm, one or more of the parameters, and 
 assigning at least one random value to each of the selected parameters; 
   a second component configured to, for each generated genome,
 train a one or more models using the generated genome, and 
 for each model trained using the generated genome,
 calculate a fitness score for the trained model at least in part by applying the trained model to a validation data set, and 
 
 produce a fitness score for the generated genome based at least in part on the fitness scores generated for the models trained using the generated genome; 
   a third component configured to identify, from among the generated genomes, a plurality of genomes having a fitness score that exceeds a fitness threshold; and   a fourth component configured to, for each of the identified genomes, mutate the identified genome,   
       wherein at least one of the components comprises computer-executable instructions stored in the memory for execution by the system. 
     
     
         2 . The system of  claim 1 , further comprising:
 a fifth component configured to, for the first genome comprising a first set of features, identify correlated features from among the first set of features at least in part by:
 for each feature of the first set of features,
 applying a feature generator associated with the feature to a training set of data to generate a feature vector for the feature, 
 
 for at least one pair of feature vectors,
 calculating a distance between each feature vector of the pair of feature vectors, 
 determining that the calculated distance is less than a distance threshold, 
 in response to determining that the calculated distance is less than a distance threshold, removing, from the first genome, a feature corresponding to at least one feature vector of the pair of feature vectors, 
 
   
       wherein each feature vector includes, for each of a plurality of patients, a single value generated by applying a first feature generator to at least one representation of physiological data representative of the patient. 
     
     
         3 . The system of  claim 2 , wherein the removing, from the first genome, of at least one feature corresponding to at least one feature vector of a first pair of feature vectors comprises:
 randomly selecting one of feature vector of the first pair of feature vectors,   identifying, from among the features of the first genome, a feature corresponding to the randomly selected feature vector; and   removing, from the first genome, the identified feature.   
     
     
         4 . The system of  claim 1 , further comprising:
 a fifth component configured to, for the first genome comprising a first set of features, generate a graph comprising a vertex for each feature of the first set of features;   a sixth component configure to generate an edge between vertices whose corresponding features have a correlation value that exceeds a correlation threshold or a distance value that is less than a distance threshold; and   a seventh component configured to remove vertices from the graph until no connected vertices remain in the graph.   
     
     
         5 . The system of  claim 1 , further comprising:
 a machine configured to receive physiological signal data from at least one patient;   a fifth component configured to, for each patient,
 apply at least one of the trained models to at least a portion of the physiological signal data received for the patient by the machine, and 
 generate a prediction for the patient based at least in part on the application of the at least one of the trained models to at least a portion of the received physiological signal. 
   
     
     
         6 . A method, performed by a computing system having a memory and a processor, for discovering machine learning genomes, the method comprising:
 generating, with the processor, a plurality of genomes, wherein each genome identifies at least one feature and at least one parameter for at least one machine learning algorithm;   for each generated genome,
 training at least one model using the generated genome, and 
 producing a fitness score for the genome based at least in part on the trained at least one model; 
   identifying, from among the generated genomes, at least one genome having a fitness score that exceeds a fitness threshold; and   mutating each identified genome.   
     
     
         7 . The method of  claim 6 , wherein generating a first genome of the plurality of genomes comprises:
 randomly selecting, from among a set of features, one or more of the features;   randomly selecting, from among a set of parameters for at least one machine learning algorithm, one or more of the parameters; and   assigning at least one value to each of the selected parameters.   
     
     
         8 . The method of  claim 7 , wherein generating the first genome further comprises:
 for each feature of the randomly selected features,
 retrieving a feature vector for the feature based at least in part on a feature generator associated with the feature and a training set of data; 
   identifying pairs of correlated feature vectors from among the generated feature vectors; and   for each identified pair of correlated feature vectors,
 identifying one feature vector of the pair of correlated feature vectors, 
 removing, from the first genome, the feature associated with the feature generator used to generate the identified feature vector; 
 randomly selecting, from among the set of features, a feature to add to the first genome, and 
 adding the randomly selected feature to the first genome. 
   
     
     
         9 . The method of  claim 8 , wherein identifying pairs of correlated feature vectors comprises:
 for each pair of feature vectors,
 calculating a distance metric for the pair of feature vectors, and 
 determining whether the distance metric calculated for the pair of feature vectors is less than a distance threshold, 
   
       wherein the distance threshold is determined based at least in part on the calculated distance metrics determined for each pair of feature vectors. 
     
     
         10 . The method of  claim 6 , wherein producing a fitness score for a first genome comprises:
 identifying a number of false positives generated by applying, to two or more validation data sets, a model trained using the first genome; and   identifying a number of false negatives generated by applying, to two or more validation data sets, a model trained using the first genome.   
     
     
         11 . The method of  claim 6 , wherein producing a fitness score for a first genome comprises:
 generating, for at least one model trained using the first genome, a receiver operating characteristic curve; and   calculating an area under the generated receiver operating characteristic curve.   
     
     
         12 . The method of  claim 6 , wherein producing a fitness score for a first genome comprises calculating, for at least one model trained using the first genome, one or more of the errors selected from the group comprising: mean squared prediction error, mean absolute error, interquartile error, and log loss error, receiver-operator characteristic curve error, and f-score error. 
     
     
         13 . The method of  claim 6 , wherein mutating a first identified genome comprises:
 selecting at least one feature of the first identified genome; and   removing, from the first identified genome, each of the selected features of the first identified genome.   
     
     
         14 . The method of  claim 6 , wherein mutating the first identified genome further comprises:
 randomly selecting, from among the set of features, a plurality of the features; and   adding, to the first identified genome, each of the randomly selected plurality of features.   
     
     
         15 . The method of  claim 6 , wherein mutating a first identified genome comprises:
 modifying at least one feature of the first identified genome.   
     
     
         16 . The method of  claim 5 , wherein mutating a first identified genome comprises:
 modifying at least one machine learning algorithm parameter of the first identified genome.   
     
     
         17 . A computer-readable medium storing instructions that, if executed by a computing system having a memory and a processor, cause the computing system to perform a method for discovering machine learning genomes, the method comprising:
 generating a plurality of genomes, wherein each genome identifies at least one feature;   for each generated genome,
 training at least one model using the generated genome, and 
 producing a fitness score for the genome based at least in part on the trained at least one model; and 
   identifying, from among the generated genomes, one or more genomes having a fitness score that exceeds a fitness threshold.   
     
     
         18 . The computer-readable medium of  claim 17 , wherein each genome further identifies at least one parameter for at least one machine learning algorithm. 
     
     
         19 . The computer-readable medium of  claim 17 , the method further comprising:
 mutating each identified genome having a fitness score that exceeds the fitness threshold.   
     
     
         20 . The computer-readable medium of  claim 17 , the method further comprising:
 computing the fitness threshold at least in part by, determining an overall fitness score based on the fitness scores produced for each of the generated genomes.   
     
     
         21 - 26 . (canceled)