Managing machine learning features

Abstract

A machine learning model is trained. A feature importance metric is determined for each machine learning feature of a plurality of machine learning features of the machine learning model. Based on the feature importance metrics, one or more machine learning features of the plurality of machine learning features of the machine learning model are managed.

What is claimed is: 
     
         1 . A method, comprising:
 training a machine learning model;   determining a feature importance metric for each machine learning feature of a plurality of machine learning features of the machine learning model; and   based on the feature importance metrics of the plurality of machine learning features, managing one or more machine learning features of the plurality of machine learning features of the machine learning model;   wherein determining the feature importance metric for a selected one of the machine learning features includes comparing a base performance of the machine learning model for a selected test dataset with a new performance of the machine learning model for a modified version of the selected test dataset of the selected machine learning feature.   
     
     
         2 . The method of  claim 1 , wherein managing the one or more machine learning features includes generating a new version of the machine learning model based on the feature importance metrics. 
     
     
         3 . The method of  claim 1 , wherein managing the one or more machine learning features includes determining to remove one of the one or more machine learning features based on a determination that its associated feature importance metric does not meet a threshold value. 
     
     
         4 . The method of  claim 1 , wherein managing the one or more machine learning features includes retraining the machine learning model to remove at least one of the one or more machine learning features from the machine learning model. 
     
     
         5 . The method of  claim 4 , wherein managing the one or more machine learning features includes automatically deleting stored data of the at least one removed feature. 
     
     
         6 . The method of  claim 4 , wherein managing the one or more machine learning features includes automatically causing data of the at least one removed feature to be no longer collected. 
     
     
         7 . The method of  claim 1 , wherein managing the one or more machine learning features includes modifying at least one of the one or more machine learning features and retraining the machine learning model using the at least one modified feature of the one or more machine learning features. 
     
     
         8 . The method of  claim 1 , wherein managing the one or more machine learning features includes generating a new feature based on at least one of the one or more machine learning features and retraining the machine learning model using the new feature. 
     
     
         9 . The method of  claim 1 , wherein the modified version of the selected test dataset of the selected machine learning feature is generated at least in part by modifying values corresponding to the selected machine learning feature using a modification approach selected based on a property of machine learning model. 
     
     
         10 . The method of  claim 1 , wherein the modified version of the selected test dataset of the selected machine learning feature is generated at least in part by randomizing values corresponding to the selected machine learning feature. 
     
     
         11 . The method of  claim 1 , wherein the each feature importance metric includes a numerical value representing an amount of contribution of the corresponding machine learning feature to an inference result of the machine learning model. 
     
     
         12 . The method of  claim 1 , wherein the each feature importance metric includes a numerical value representing a rank order of the corresponding machine learning feature as compared to others of the machine learning features. 
     
     
         13 . A system, comprising:
 one or more processors configured to:
 train a machine learning model; 
 determine a feature importance metric for each machine learning feature of a plurality of machine learning features of the machine learning model; and 
 based on the feature importance metrics of the plurality of machine learning features, manage one or more machine learning features of the plurality of machine learning features of the machine learning model, wherein being configured to determine the feature importance metric for a selected one of the machine learning features includes being configured to compare a base performance of the machine learning model for a selected test dataset with a new performance of the machine learning model for a modified version of the selected test dataset of the selected machine learning feature; and 
   a memory coupled to at least one of the one or more processors and configured to provide the at least one of the one or more processors with instructions.   
     
     
         14 . The system of  claim 13 , wherein being configured to manage the one or more machine learning features includes being configured to determine to remove one of the one or more machine learning features based on a determination that its associated feature importance metric does not meet a threshold value. 
     
     
         15 . The system of  claim 13 , wherein being configured to manage the one or more machine learning features includes being configured to retrain the machine learning model to remove at least one of the one or more machine learning features from the machine learning model. 
     
     
         16 . The system of  claim 13 , wherein being configured to manage the one or more machine learning features includes being configured to generate a new feature based on at least one of the one or more machine learning features and retraining the machine learning model using the new feature. 
     
     
         17 . The system of  claim 13 , wherein the modified version of the selected test dataset of the selected machine learning feature is generated at least in part by modifying values corresponding to the selected machine learning feature using a modification approach selected based on a property of machine learning model. 
     
     
         18 . The system of  claim 13 , wherein the modified version of the selected test dataset of the selected machine learning feature is generated at least in part by randomizing values corresponding to the selected machine learning feature. 
     
     
         19 . The system of  claim 13 , wherein the each feature importance metric includes a numerical value representing an amount of contribution of the corresponding machine learning feature to an inference result of the machine learning model. 
     
     
         20 . A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
 training a machine learning model;   determining a feature importance metric for each machine learning feature of a plurality of machine learning features of the machine learning model; and   based on the feature importance metrics of the plurality of machine learning features, managing one or more machine learning features of the plurality of machine learning features of the machine learning model;   wherein determining the feature importance metric for a selected one of the machine learning features includes comparing a base performance of the machine learning model for a selected test dataset with a new performance of the machine learning model for a modified version of the selected test dataset of the selected machine learning feature.