Pipeline modeler supporting attribution analysis

Abstract

Techniques are disclosed for attribution analysis in analytical workflows. A data processing system (DPS) obtains an overall model comprising one or more sub-models. The DPS selects an output variable of the overall model for which attribution of changes is to be performed, and a plurality of input variables against which changes are to be attributed to. The overall model is initially executed with respect to a data set of values for the plurality of input variables to generate a base result for the output variable. The overall model is iteratively executed based on a condition associated with the plurality of input variables to obtain a new result for the output variable. In each iteration, a value of an input variable is changed with respect to the data set of values and a change in the output variable with respect to the base result is attributed to the corresponding input variable.

What is claimed: 
     
       1. A method comprising:
 obtaining, by a data processing system, an overall model comprising one or more sub-models that are executed in a predetermined order to determine at least one output variable of the overall model; 
 selecting, by the data processing system, an output variable of the overall model for which attribution of changes is to be performed, and a plurality of input variables against which changes are to be attributed to; 
 initially executing, by the data processing system, the overall model by: (i) obtaining a data set of values for the plurality of input variables, (ii) executing the one or more sub-models in the predetermined order based on the data set of values for the plurality of input variables, and (iii) generating a base result for the output variable of the overall model based on the executing of the one or more sub-models; 
 iteratively executing, by the data processing system, the overall model, wherein each iterative execution of the overall model comprises: (i) selecting an input variable from the plurality of input variables, wherein the input variable selected is different in each iteration, (ii) generating a new data set of values for the plurality of input variables, wherein a value of the input variable in the new data set of values is different that the value of the input variable in the data set of values for the plurality of input variable, and values of each remaining input variable in the new data set of values is same as the values of each remaining input variables in the data set of values for the plurality of input variables, (iii) executing the one or more sub-models in the predetermined order based on the new data set of values for the plurality of input variable, (iv) generating a new result for the output variable of the overall model based on the executing of the one or more sub-models, (v) determining a difference between the base result and the new result, and (vi) attributing the difference to the input variable selected; and 
 generating a report to be provided to a user, wherein the report includes change in value of each input variable of the plurality of input variables, and associated change in the output variable of the overall model. 
 
     
     
       2. The method of  claim 1 , wherein the overall model is executed iteratively N times, wherein N is a number of input variables of the plurality of input variables, and the output variable is a function of the plurality of input variables. 
     
     
       3. The method of  claim 1 , wherein executing the one or more sub-models in the predetermined order includes executing a first subset of the one or more sub-models concurrently, and executing a second subset of the one or more sub-models sequentially. 
     
     
       4. The method of  claim 1 , wherein the data set of values for the plurality of input variables is obtained via an application programming interface from one or more data sources external to the data processing system. 
     
     
       5. The method of  claim 1 , wherein initially executing, by the data processing system, the overall model further comprises:
 obtaining one or more reference data sets of values for the plurality of input variables; 
 executing the one or more sub-models with respect to each reference data set of values for the plurality of input variables to obtain a reference result of the output variable of the overall model; 
 computing an average reference result based on reference results corresponding to the one or more reference data sets of values for the plurality of input variables; 
 determining a first difference between the base result and the average reference result; and 
 utilizing the data set of values for the plurality of input variables in initial execution of the overall model in response to the first difference being less than a threshold value. 
 
     
     
       6. The method of  claim 1 , wherein iteratively executing, by the data processing system, the overall model further comprises:
 obtaining, via an application programming interface, a first change in an output value of at least one sub-model with respect to the new data set of values for the plurality of input variables. 
 
     
     
       7. The method of  claim 6 , further comprising:
 determining whether the first change in the output value of at least one sub-model is greater than a threshold value; and 
 responsive to the first change being greater than the threshold value, generating a message to be transmitted to the user, the message indicating that a re-training of the at least one sub-model is to be performed. 
 
     
     
       8. A computing device comprising:
 a processor; and 
 a memory including instructions that, when executed with the processor, cause the computing device to, at least:
 obtain an overall model comprising one or more sub-models that are executed in a predetermined order to determine at least one output variable of the overall model; 
 select an output variable of the overall model for which attribution of changes is to be performed, and a plurality of input variables against which changes are to be attributed to; 
 initially execute the overall model by: (i) obtaining a data set of values for the plurality of input variables, (ii) executing the one or more sub-models in the predetermined order based on the data set of values for the plurality of input variables, and (iii) generating a base result for the output variable of the overall model based on the executing of the one or more sub-models; 
 iteratively execute the overall model, wherein each iterative execution of the overall model comprises: (i) selecting an input variable from the plurality of input variables, wherein the input variable selected is different in each iteration, (ii) generating a new data set of values for the plurality of input variables, wherein a value of the input variable in the new data set of values is different that the value of the input variable in the data set of values for the plurality of input variable, and values of each remaining input variables in the new data set of values is same as the values of each remaining input variable in the data set of values for the plurality of input variable, (iii) executing the one or more sub-models in the predetermined order based on the new data set of values for the plurality of input variable, (iv) generating a new result for the output variable of the overall model based on the executing of the one or more sub-models, (v) determining a difference between the base result and the new result, and (vi) attributing the difference to the input variable selected; and 
 generate a report to be provided to a user, wherein the report includes change in value of each input variable of the plurality of input variables, and associated change in the output variable of the overall model. 
 
 
     
     
       9. The computing device of  claim 8 , wherein the overall model is executed iteratively N times, wherein N is a number of input variables of the plurality of input variables, and the output variable is a function of the plurality of input variables. 
     
     
       10. The computing device of  claim 8 , wherein executing the one or more sub-models in the predetermined order includes executing a first subset of the one or more sub-models concurrently, and executing a second subset of the one or more sub-models sequentially. 
     
     
       11. The computing device of  claim 8 , wherein the data set of values for the plurality of input variables is obtained via an application programming interface from one or more external data sources. 
     
     
       12. The computing device of  claim 8 , wherein the processor is further configured to:
 obtain one or more reference data sets of values for the plurality of input variables; 
 execute the one or more sub-models with respect to each reference data set of values for the plurality of input variables to obtain a reference result of the output variable of the overall model; 
 compute an average reference result based on reference results corresponding to the one or more reference data sets of values for the plurality of input variables; 
 determine a first difference between the base result and the average reference result; and 
 utilize the data set of values for the plurality of input variables in initial execution of the overall model in response to the first difference being less than a threshold value. 
 
     
     
       13. The computing device of  claim 8 , wherein the processor is further configured to:
 obtain, via an application programming interface, a first change in an output value of at least one sub-model with respect to the new data set of values for the plurality of input variables. 
 
     
     
       14. The computing device of  claim 13 , wherein the processor is further configured to:
 determine whether the first change in the output value of at least one sub-model is greater than a threshold value; and 
 responsive to the first change being greater than the threshold value, generate a message to be transmitted to the user, the message indicating that a re-training of the at least one sub-model is to be performed. 
 
     
     
       15. A non-transitory computer readable medium storing specific computer-executable instructions that, when executed by a processor, cause a computer system to at least:
 obtain an overall model comprising one or more sub-models that are executed in a predetermined order to determine at least one output variable of the overall model; 
 select an output variable of the overall model for which attribution of changes is to be performed, and a plurality of input variables against which changes are to be attributed to; 
 initially execute the overall model by: (i) obtaining a data set of values for the plurality of input variables, (ii) executing the one or more sub-models in the predetermined order based on the data set of values for the plurality of input variables, and (iii) generating a base result for the output variable of the overall model based on the executing of the one or more sub-models; 
 iteratively execute the overall model, wherein each iterative execution of the overall model comprises: (i) selecting an input variable from the plurality of input variables, wherein the input variable selected is different in each iteration, (ii) generating a new data set of values for the plurality of input variables, wherein a value of the input variable in the new data set of values is different that the value of the input variable in the data set of values for the plurality of input variable, and values of each remaining input variables in the new data set of values is same as the values of each remaining input variable in the data set of values for the plurality of input variable, (iii) executing the one or more sub-models in the predetermined order based on the new data set of values for the plurality of input variable, (iv) generating a new result for the output variable of the overall model based on the executing of the one or more sub-models, (v) determining a difference between the base result and the new result, and (vi) attributing the difference to the input variable selected; and 
 generate a report to be provided to a user, wherein the report includes change in value of each input variable of the plurality of input variables, and associated change in the output variable of the overall model. 
 
     
     
       16. The non-transitory computer readable medium of  claim 15 , wherein the overall model is executed iteratively N times, wherein N is a number of input variables of the plurality of input variables, and the output variable is a function of the plurality of input variables. 
     
     
       17. The non-transitory computer readable medium of  claim 15 , wherein the data set of values for the plurality of input variables is obtained via an application programming interface from one or more external data sources. 
     
     
       18. The non-transitory computer readable medium of  claim 15 , wherein the computer system is further configured to:
 obtain one or more reference data sets of values for the plurality of input variables; 
 execute the one or more sub-models with respect to each reference data set of values for the plurality of input variables to obtain a reference result of the output variable of the overall model; 
 compute an average reference result based on reference results corresponding to the one or more reference data sets of values for the plurality of input variables; 
 determine a first difference between the base result and the average reference result; and 
 utilize the data set of values for the plurality of input variables in initial execution of the overall model in response to the first difference being less than a threshold value. 
 
     
     
       19. The non-transitory computer readable medium of  claim 15 , wherein the computer system is further configured to:
 obtain, via an application programming interface, a first change in an output value of at least one sub-model with respect to the new data set of values for the plurality of input variables. 
 
     
     
       20. The non-transitory computer readable medium of  claim 19 , wherein the computer system is further configured to:
 determine whether the first change in the output value of at least one sub-model is greater than a threshold value; and 
 responsive to the first change being greater than the threshold value, generate a message to be transmitted to the user, the message indicating that a re-training of the at least one sub-model is to be performed.