Simultaneous data sampling and feature selection via weak learners

Abstract

From many features and many multidimensional points, a computer generates exploratory training configurations. Each point contains a value for each of the features. Each exploratory training configuration identifies a random subset of the features and a random subset of the points. A performance score is generated for each of the exploratory training configurations. A feature weight is generated for each of the features that is based on the performance scores of the exploratory training configurations whose random subset of features contains the feature. A point weight is generated for each of the points that is based on the performance scores of the exploratory training configurations whose random subset of the many points contains the point. A machine learning model is trained using an optimized training corpus that consists of a subset of the many features based on feature weight and a subset of the many points based on point weight.

What is claimed is: 
     
         1 . A method comprising:
 generating, from a plurality of features and a plurality of multidimensional points, a plurality of training configurations wherein:
 each point in the plurality of multidimensional points contains a value for each feature in the plurality of features, and 
 each configuration in the plurality of training configurations identifies a random subset of the plurality of features and a random subset of the plurality of multidimensional points; 
   generating a performance score for each configuration in the plurality of training configurations;   generating a feature weight for each feature in the plurality of features that is based on the performance scores of the plurality of training configurations whose random subset of the plurality of features contains the feature;   generating a point weight for each point in the plurality of multidimensional points that is based on the performance scores of the plurality of training configurations whose random subset of the plurality of multidimensional points contains the point; and   training a machine learning model based on:
 a subset of the plurality of features based on feature weight, and 
 a subset of the plurality of multidimensional points based on point weight; 
   wherein the method is performed by one or more computers.   
     
     
         2 . The method of  claim 1  wherein said generating the performance score for each configuration in the plurality of training configurations comprises training a weak learner based on the random subset of the plurality of features of the configuration and the random subset of the plurality of multidimensional points of the configuration. 
     
     
         3 . The method of  claim 2  wherein:
 the weak learner comprises a decision tree; and 
 the machine learning model does not comprise a decision tree. 
 
     
     
         4 . The method of  claim 1  further comprising for each feature in the plurality of features, counting how many of the plurality of training configurations whose random subset of the plurality of features of the configuration contains the feature. 
     
     
         5 . The method of  claim 1  further comprising for each point in the plurality of multidimensional points, counting how many of the plurality of training configurations whose random subset of the plurality of multidimensional points of the configuration contains the point. 
     
     
         6 . The method of  claim 1  wherein said generating the plurality of training configurations comprises generating a fixed count of training configurations. 
     
     
         7 . The method of  claim 1  further comprising for each feature in the plurality of features, summing performance scores of the plurality of training configurations whose random subset of the plurality of features of the configuration contains the feature. 
     
     
         8 . The method of  claim 1  further comprising for each point in the plurality of multidimensional points, summing performance scores of the plurality of training configurations whose random subset of the plurality of multidimensional points of the configuration contains the point. 
     
     
         9 . The method of  claim 1  wherein at least one selected from a group consisting of:
 said generating the plurality of training configurations occurs before said generating the performance score for each configuration in the plurality of training configurations, and 
 said generating the plurality of training configurations does not depend on said generating the performance score for each configuration in the plurality of training configurations. 
 
     
     
         10 . The method of  claim 1  wherein said training the machine learning model comprises the machine learning model accepting as input a point weight of a point in the plurality of multidimensional points. 
     
     
         11 . One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause:
 generating, from a plurality of features and a plurality of multidimensional points, a plurality of training configurations wherein:
 each point in the plurality of multidimensional points contains a value for each feature in the plurality of features, and 
 each configuration in the plurality of training configurations identifies a random subset of the plurality of features and a random subset of the plurality of multidimensional points; 
   generating a performance score for each configuration in the plurality of training configurations;   generating a feature weight for each feature in the plurality of features that is based on the performance scores of the plurality of training configurations whose random subset of the plurality of features contains the feature;   generating a point weight for each point in the plurality of multidimensional points that is based on the performance scores of the plurality of training configurations whose random subset of the plurality of multidimensional points contains the point; and   training a machine learning model based on:
 a subset of the plurality of features based on feature weight, and 
 a subset of the plurality of multidimensional points based on point weight. 
   
     
     
         12 . The one or more non-transitory computer-readable media of  claim 11  wherein said generating the performance score for each configuration in the plurality of training configurations comprises training a weak learner based on the random subset of the plurality of features of the configuration and the random subset of the plurality of multidimensional points of the configuration. 
     
     
         13 . The one or more non-transitory computer-readable media of  claim 12  wherein:
 the weak learner comprises a decision tree; and 
 the machine learning model does not comprise a decision tree. 
 
     
     
         14 . The one or more non-transitory computer-readable media of  claim 11  wherein the instructions further cause for each feature in the plurality of features, counting how many of the plurality of training configurations whose random subset of the plurality of features of the configuration contains the feature. 
     
     
         15 . The one or more non-transitory computer-readable media of  claim 11  wherein the instructions further cause for each point in the plurality of multidimensional points, counting how many of the plurality of training configurations whose random subset of the plurality of multidimensional points of the configuration contains the point. 
     
     
         16 . The one or more non-transitory computer-readable media of  claim 11  wherein said generating the plurality of training configurations comprises generating a fixed count of training configurations. 
     
     
         17 . The one or more non-transitory computer-readable media of  claim 11  wherein the instructions further cause for each feature in the plurality of features, summing performance scores of the plurality of training configurations whose random subset of the plurality of features of the configuration contains the feature. 
     
     
         18 . The one or more non-transitory computer-readable media of  claim 11  wherein the instructions further cause for each point in the plurality of multidimensional points, summing performance scores of the plurality of training configurations whose random subset of the plurality of multidimensional points of the configuration contains the point. 
     
     
         19 . The one or more non-transitory computer-readable media of  claim 11  wherein at least one selected from a group consisting of:
 said generating the plurality of training configurations occurs before said generating the performance score for each configuration in the plurality of training configurations, and 
 said generating the plurality of training configurations does not depend on said generating the performance score for each configuration in the plurality of training configurations. 
 
     
     
         20 . The one or more non-transitory computer-readable media of  claim 11  wherein said training the machine learning model comprises the machine learning model accepting as input a point weight of a point in the plurality of multidimensional points.