Systems and methods for improving machine learning models via dimensionality reduction evaluation to reduce disparate impact on protected class individuals

Abstract

Systems and methods for model evaluation. A protected class model that satisfies an accuracy threshold is built by using: data sets for use by a modeling system being evaluated, and protected class membership information for each data set. A target for the protected class model is a protected class membership variable indicating membership in a protected class. Each predictor of the protected class model is a predictor of an evaluated model used by the modeling system. A target of the evaluated model is different from the target of the protected class model. Each predictor is a set of one or more variables of the data sets. For each predictor of the protected class model, a protected class model impact ranking value and a modeling system impact ranking value are determined.

What is claimed is: 
     
         1 . A method of improving machine learning models via dimensionality reduction evaluation, the method implemented by a model evaluation system and comprising:
 using predictors used by a second machine learning model hosted by a decisioning system communicably coupled to the model evaluation system via one or more networks, input rows used by the second machine learning model, and protected class membership information for individuals to determine a set of the predictors that results in a first machine learning model that satisfies an accuracy threshold, wherein the first machine learning models are trained to predict membership in a protected class and each of the predictors is a variable of one or more of the input rows;   for each predictor of the set of the predictors, determining a first ranking value and a second ranking value, wherein the first ranking value indicates relative influence on a prediction generated by the first machine learning model and the second ranking value is determined by providing one or more first requests to the decisioning system to cause the decisioning system to generate and return a first decisioning output for at least one of the input rows by executing the second machine learning model;   providing a second request to the decisioning system to cause the decisioning system to modify the second machine learning model to remove one of the set of the predictors for which the second ranking value exceeds a first threshold and the first ranking value exceeds a second threshold; and   instructing the decisioning system to start operation of the second modified machine learning model in a production environment to generate decisioning outputs responsive to third requests from user devices and with reduced disproportionate impact on protected class members as compared to the second machine learning model.   
     
     
         2 . The method of  claim 1 , wherein the modified second machine learning model comprises a credit underwriting model for facilitating credit decisions and the decisioning outputs are indicative of credit worthiness of persons associated with the evaluation requests. 
     
     
         3 . The method of  claim 1 , further comprising providing an operator device with an indication that the second machine learning model satisfies credit decision fairness requirements when another machine learning model that satisfies the accuracy threshold cannot be built. 
     
     
         4 . The method of  claim 1 , further comprising generating the protected class membership information using another set of the predictors of the input rows or a Bayesian improved surname geocoding (BISG) process. 
     
     
         5 . The method of  claim 1 , wherein the second request comprises an indication of another variable corresponding to the identified one of the set of the predictors and the second request causes the decisioning system to modify the second machine learning model to remove the other variable from the second machine learning model to thereby reduce a likelihood the second machine learning model has a discriminatory output. 
     
     
         6 . The method of  claim 1 , further comprising generating and outputting at least one visualization comprising a graphical representation of the relative influence, wherein the visualization comprises a heat map, self-organizing map (SOM), self-organizing feature map, kahonen map, tree, histogram, or chart. 
     
     
         7 . The method of  claim 1 , further comprising:
 for each of the predictors, determining a protected class model impact value for a first population of the input rows and a decisioning system impact value for the first population of the input rows;   generating a first sorted list that includes identifiers for each of the predictors ordered according to the protected class model impact values and a second sorted list that includes identifiers for each of the predictors ordered according to the decisioning system impact values; and   for each of the predictors, determining the first ranking value based on an order of the predictor identifier in the first sorted list the second ranking value based on an order of the predictor identifier in the second sorted list.   
     
     
         8 . The method of  claim 7 , further comprising:
 for each of the input rows of the first population, generating an input row decisioning system impact value for each of the predictors; and   for each of the predictors, determining the decisioning system impact value for the first population of the input rows using the input row decisioning system impact values, wherein each input row decisioning system impact value indicates another relative influence of a corresponding one of the predictors on a second decisioning output generated by the decisioning system for a corresponding one of the input rows.   
     
     
         9 . The method of  claim 7 , further comprising generating at least one modified input row by modifying data of one of the input rows, controlling the decisioning system to generate a third decisioning output for at least one modified input row, comparing the second decisioning output for one of the input rows to the third decisioning output for the at least one modified input row, and generating the input row decisioning system impact value based on a result of the comparison. 
     
     
         10 . The method of  claim 7 , further comprising determining an output decomposition comprising one or more of using gradient values provided by the decisioning system, accessing tree node information, or generating Shapley values. 
     
     
         11 . The method of  claim 1 , wherein the protected class includes at least one of a person's race, color, religion, national origin, sex, age, or disability status. 
     
     
         12 . A model evaluation system, comprising memory comprising instructions stored thereon and one or more processors coupled to the memory and configured to execute the stored instructions to:
 use predictors used by a second machine learning model hosted by a decisioning system communicably coupled to the model evaluation system via one or more networks, input rows used by the second machine learning model, and protected class membership information for individuals to determine a set of the predictors that results in a first machine learning model that satisfies an accuracy threshold, wherein the first machine learning models are trained to predict membership in a protected class and each of the predictors is a first variable of one or more of the input rows;   for each predictor of the set of the predictors, determine a first ranking value and a second ranking value, wherein the first ranking value indicates relative influence on a prediction generated by the first machine learning model and the second ranking value is determined by providing one or more first requests to the decisioning system to cause the decisioning system to generate and return a first decisioning output for at least one of the input rows by executing the second machine learning model;   provide a second request to the decisioning system to cause the decisioning system to modify the second machine learning model to remove one of the set of the predictors for which the second ranking value exceeds a first threshold and the first ranking value exceeds a second threshold; and   instruct the decisioning system to start operation of the modified second machine learning model in a production environment to generate decisioning outputs responsive to third requests from user devices and with reduced disproportionate impact on protected class members as compared to the second machine learning model.   
     
     
         13 . The model evaluation system of  claim 12 , wherein the protected class includes at least one of a person's race, color, religion, national origin, sex, age, or disability status and the decisioning outputs are indicative of credit worthiness of persons associated with the third requests. 
     
     
         14 . The model evaluation system of  claim 12 , wherein the one or more processors are further configured to execute the stored instructions to provide an operator device with information indicating that the second machine learning model satisfies credit decision fairness requirements when another first machine learning model that satisfies the accuracy threshold cannot be built. 
     
     
         15 . The model evaluation system of  claim 12 , wherein the one or more processors are further configured to execute the stored instructions to generate the protected class membership information using another set of the predictors of the input rows or a Bayesian improved surname geocoding (BISG) process. 
     
     
         16 . The model evaluation system of  claim 12 , wherein the second request comprises an indication of a second variable corresponding to the one of the set of the predictors and the second request causes the decisioning system to modify the second machine learning model to remove the second variable from the second machine learning model to thereby reduce a likelihood the second machine learning model has a discriminatory output. 
     
     
         17 . The model evaluation system of  claim 12 , wherein the one or more processors are further configured to execute the stored instructions to generate and output at least one visualization comprising a graphical representation of the relative influence, wherein the visualization comprises a heat map, self-organizing map (SOM), self-organizing feature map, kahonen map, tree, histogram, or chart. 
     
     
         18 . The model evaluation system of  claim 12 , wherein the one or more processors are further configured to execute the stored instructions to:
 for each of the predictors, determine a protected class model impact value for a first population of the input rows and a decisioning system impact value for the first population of the input rows;   generate a first sorted list that includes identifiers for each of the predictors ordered according to the protected class model impact values and a second sorted list that includes identifiers for each of the predictors ordered according to the decisioning system impact values; and   for each of the predictors, determine the first ranking value based on an order of the predictor identifier in the first sorted list the second ranking value based on an order of the predictor identifier in the second sorted list.   
     
     
         19 . The model evaluation system of  claim 18 , wherein the one or more processors are further configured to execute the stored instructions to:
 for each of the input rows of the first population, generate an input row decisioning system impact value for each of the predictors; and   for each of the predictors, determine the decisioning system impact value for the first population of the input rows using the input row decisioning system impact values, wherein each input row decisioning system impact value indicates another relative influence of a corresponding one of the predictors on a second decisioning output generated by the decisioning system for a corresponding one of the input rows.   
     
     
         20 . The model evaluation system of  claim 18 , wherein the one or more processors are further configured to execute the stored instructions to generate at least one modified input row by modifying data of one of the input rows, control the decisioning system to generate a third decisioning output for at least one modified input row, comparing the second decisioning output for one of the input rows to the third decisioning output for the at least one modified input row, and generating the input row decisioning system impact value based on a result of the comparison. 
     
     
         21 . The model evaluation system of  claim 18 , wherein the one or more processors are further configured to execute the stored instructions to determine an output decomposition comprising one or more of using gradient values provided by the decisioning system, accessing tree node information, or generating Shapley values. 
     
     
         22 . A non-transitory computer readable medium having stored thereon instructions comprising executable code that, when executed by one or more processors, causes the one or more processors to:
 use predictors used by a second machine learning model hosted by a decisioning system communicably coupled to the model evaluation system via one or more networks, input rows used by the second machine learning model, and protected class membership information for individuals to determine a set of the predictors that results in a first machine learning model that satisfies an accuracy threshold, wherein the first machine learning models are trained to predict membership in a protected class and each of the predictors is a first variable of one or more of the input rows;   for each predictor of the set of the predictors, determine a first ranking value and a second ranking value, wherein the first ranking value indicates relative influence on a prediction generated by the first machine learning model and the second ranking value is determined by providing one or more first requests to the decisioning system to cause the decisioning system to generate and return a first decisioning output for at least one of the input rows by executing the second machine learning model;   provide a second request to the decisioning system to cause the decisioning system to modify the second machine learning model to remove one of the set of the predictors for which the second ranking value exceeds a first threshold and the first ranking value exceeds a second threshold; and   instruct the decisioning system to start operation of the modified second machine learning model in a production environment to generate decisioning outputs responsive to third requests from user devices and with reduced disproportionate impact on protected class members as compared to the second machine learning model.   
     
     
         23 . The non-transitory computer readable medium of  claim 22 , wherein the executable code, when executed by one or more processors, further causes one or more processors to provide an operator device with information indicating that the second machine learning model satisfies credit decision fairness requirements when another machine learning model that satisfies the accuracy threshold cannot be built. 
     
     
         24 . The non-transitory computer readable medium of  claim 22 , wherein the executable code, when executed by one or more processors, further causes one or more processors to generate the protected class membership information using another set of the predictors of the input rows or a Bayesian improved surname geocoding (BISG) process. 
     
     
         25 . The non-transitory computer readable medium of  claim 22 , wherein the second request comprises an indication of a second variable corresponding to the identified one of the set of the predictors and the second request causes the decisioning system to modify the second machine learning model to remove the second variable from the second machine learning model to thereby reduce a likelihood the second machine learning model has a discriminatory output. 
     
     
         26 . The non-transitory computer readable medium of  claim 22 , wherein the executable code, when executed by one or more processors, further causes one or more processors to generate and output at least one visualization comprising a graphical representation of the relative influence, wherein the visualization comprises a heat map, self-organizing map (SOM), self-organizing feature map, kahonen map, tree, histogram, or chart. 
     
     
         27 . The non-transitory computer readable medium of  claim 22 , wherein the executable code, when executed by one or more processors, further causes the one or more processors to:
 for each of the predictors, determine a protected class model impact value for a first population of the input rows and a decisioning system impact value for the first population of the input rows;   generate a first sorted list that includes identifiers for each of the predictors ordered according to the protected class model impact values and a second sorted list that includes identifiers for each of the predictors ordered according to the decisioning system impact values; and   for each of the predictors, determine the first ranking value based on an order of the predictor identifier in the first sorted list the second ranking value based on an order of the predictor identifier in the second sorted list.   
     
     
         28 . The non-transitory computer readable medium of  claim 27 , wherein the executable code, when executed by one or more processors, further causes the one or more processors to:
 for each of the input rows of the first population, generate an input row decisioning system impact value for each of the predictors; and   for each of the predictors, determine the decisioning system impact value for the first population of the input rows using the input row decisioning system impact values, wherein each input row decisioning system impact value indicates another relative influence of a corresponding one of the predictors on a second decisioning output generated by the decisioning system for a corresponding one of the input rows.   
     
     
         29 . The non-transitory computer readable medium of  claim 27 , wherein the executable code, when executed by one or more processors, further causes one or more processors to generate at least one modified input row by modifying data of one of the input rows, control the decisioning system to generate a third decisioning output for the at least one modified input row, compare the second decisioning output for one of the input rows to the other decisioning output for the at least one modified input row, and generate the input row decisioning system impact value based on a result of the comparison. 
     
     
         30 . The non-transitory computer readable medium of  claim 27 , wherein the executable code, when executed by the one or more processors, further causes the one or more processors to determine an output decomposition comprising one or more of using gradient values provided by the decisioning system, accessing tree node information, or generating Shapley values.