Interactive interfaces for machine learning model evaluations

Abstract

A first data set corresponding to an evaluation run of a model is generated at a machine learning service for display via an interactive interface. The data set includes a prediction quality metric. A target value of an interpretation threshold associated with the model is determined based on a detection of a particular client's interaction with the interface. An indication of a change to the prediction quality metric that results from the selection of the target value may be initiated.

1 .- 24 . (canceled) 
     
     
         25 . A method, comprising:
 performing, at one or more computing devices:
 causing to be presented, via one or more graphical interfaces, an indication of a distribution of one or more values obtained from one or more machine learning models; 
 obtaining an indication of a requested change to a threshold associated with the one or more values; and 
 causing the indication of the distribution to be dynamically updated in accordance with the requested change. 
   
     
     
         26 . The method as recited in  claim 25 , wherein the indication of the distribution comprises a confusion matrix. 
     
     
         27 . The method as recited in  claim 25 , wherein the threshold indicates a cutoff boundary between a pair of classes. 
     
     
         28 . The method as recited in  claim 25 , wherein the threshold comprises a prediction probability threshold for inclusion of a record in a class. 
     
     
         29 . The method as recited in  claim 25 , wherein the one or more values represent an error metric. 
     
     
         30 . The method as recited in  claim 25 , further comprising performing, at the one or more computing devices:
 causing to be presented, via the one or more graphical interfaces, an indication of a plurality of alternative error metrics applicable to predictions of the one or more machine learning models; and   obtaining an indication of a particular error metric selected from the plurality of alternative error metrics, wherein the distribution of the one or more values includes a distribution of the particular error metric.   
     
     
         31 . The method as recited in  claim 25 , further comprising performing, at the one or more computing devices:
 causing to be presented, via the one or more graphical user interfaces, an indication of a plurality of alternative error metrics applicable to predictions of the one or more machine learning models; and   obtaining, via the one or more graphical interfaces, an indication of another error metric which is not included in the plurality of alternative error metrics, wherein the distribution of the one or more values includes a distribution of the other error metric.   
     
     
         32 . A system, comprising:
 one or more computing devices;   wherein the one or more computing devices include instructions that upon execution on or across one or more processors cause the one or more computing devices to:
 cause to be displayed, via one or more graphical interfaces, an indication of a distribution of one or more values obtained from one or more machine learning models; 
 obtain an indication of a requested change to a threshold associated with the one or more values; and 
 cause the indication of the distribution to be dynamically updated in accordance with the requested change. 
   
     
     
         33 . The system as recited in  claim 32 , wherein the indication of the distribution comprises a confusion matrix. 
     
     
         34 . The system as recited in  claim 32 , wherein the threshold indicates a cutoff boundary between a pair of classes. 
     
     
         35 . The system as recited in  claim 32 , wherein the threshold comprises a prediction probability threshold for inclusion of a record in a class. 
     
     
         36 . The system as recited in  claim 32 , wherein the one or more values represent an error metric. 
     
     
         37 . The system as recited in  claim 32 , wherein the one or more computing devices include further instructions that upon execution on or across one or more processors further cause the one or more computing devices to:
 cause to be displayed, via the one or more graphical interfaces, an indication of a plurality of alternative error metrics applicable to predictions of the one or more machine learning models; and   obtain an indication of a particular error metric selected from the plurality of alternative error metrics, wherein the distribution of the one or more values includes a distribution of the particular error metric.   
     
     
         38 . The system as recited in  claim 32 , wherein the indication of the requested change to the threshold is obtained based at least in part on input received via a slider control element of the one or more graphical interfaces. 
     
     
         39 . One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors cause one or more computer systems to:
 cause to be displayed, via one or more graphical interfaces, an indication of a distribution of one or more values obtained from one or more machine learning models;   obtain an indication of a requested change to a threshold associated with the one or more values; and   cause the indication of the distribution to be dynamically updated in accordance with the requested change.   
     
     
         40 . The one or more non-transitory computer-accessible storage media as recited in  claim 39 , wherein the indication of the distribution comprises a confusion matrix. 
     
     
         41 . The one or more non-transitory computer-accessible storage media as recited in  claim 39 , wherein the threshold indicates a cutoff boundary between a pair of classes. 
     
     
         42 . The one or more non-transitory computer-accessible storage media as recited in  claim 39 , wherein the threshold comprises a prediction probability threshold for inclusion of a record in a class. 
     
     
         43 . The one or more non-transitory computer-accessible storage media as recited in  claim 39 , wherein the one or more values represent an error metric. 
     
     
         44 . The one or more non-transitory computer-accessible storage media as recited in  claim 39 , storing further program instructions that when executed on or across the one or more processors further cause the one or more computer systems to:
 cause to be displayed, via the one or more graphical interfaces, an indication of a plurality of alternative error metrics applicable to predictions of the one or more machine learning models; and   obtain an indication of a particular error metric selected from the plurality of alternative error metrics, wherein the distribution of the one or more values includes a distribution of the particular error metric.