Medical scan interface feature evaluating system and methods for use therewith

Abstract

A medical scan interface feature evaluator system is operable to receive a set of responses from each of a set of client devices, where each set of responses is generated based on a corresponding client device displaying each of the set of medical scans in conjunction with at least one interface feature indicated in an image-to-prompt mapping. Response score data is generated for each response of the set of responses received from each of the set of client devices by comparing each response to truth annotation data of a corresponding medical scan of the set of medical scans indicated by the image-to-prompt mapping. Interface feature score data corresponding to each user interface feature in the set of user interface features is generated based on aggregating corresponding response score data. A ranking of the set of user interface features is generated based on the interface feature score data.

What is claimed is: 
     
         1 . A medical scan interface feature evaluator system, comprising:
 a processing system that includes a processor; and   a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations comprising:
 determining a first image-to-prompt mapping based on at least one of a set of user interface features to be displayed with each medical scan of a first set of medical scans; 
 transmitting the first set of medical scans and the first image-to-prompt mapping to a first set of client devices; 
 receiving a first set of responses from each of the first set of client devices, wherein each first set of responses is generated based on a corresponding one of the first set of client devices displaying each of the first set of medical scans in conjunction with a mapped at least one of the set of user interface features indicated in the first image-to-prompt mapping; 
 generating first response score data for each response of the first set of responses received from each of the first set of client devices by comparing each response to truth annotation data of a corresponding medical scan of the first set of medical scans indicated by the first image-to-prompt mapping; 
 generating interface feature score data corresponding to each user interface feature in the set of user interface features based on aggregating corresponding first response score data; and 
 generating a ranking of the set of user interface features based on the interface feature score data. 
   
     
     
         2 . The medical scan interface feature evaluator system of  claim 1 , wherein at least one response in the first set of responses indicates that a corresponding medical scan is normal in response to one of the first set of client devices displaying one of the set of user interface features that includes a prompt to select one of: that a displayed medical scan is normal or that the displayed medical scan includes an abnormality, and wherein generating the first response score data for the at least one response includes assigning a low score to the one response based on determining that the truth annotation data of an at least one corresponding medical scan indicates at least one abnormality. 
     
     
         3 . The medical scan interface feature evaluator system of  claim 1 , wherein at least one response in the first set of responses includes abnormality classification data generated by one of the first set of client devices in response to displaying one of the set of user interface features that includes a prompt to classify an abnormality, and wherein the generating the first response score data of the at least one response includes comparing the abnormality classification data of the at least one response to the truth annotation data of an at least one corresponding medical scan. 
     
     
         4 . The medical scan interface feature evaluator system of  claim 1 , wherein at least one response in the first set of responses includes an identified region generated by one of the first set of client devices in response to displaying one of the set of user interface features that includes a prompt to identify a region of the displayed medical scan that includes an abnormality, and wherein the generating the first response score data includes comparing the identified region of the at least one response to the truth annotation data of an at least one corresponding medical scan. 
     
     
         5 . The medical scan interface feature evaluator system of  claim 1 , wherein the operations further comprise:
 selecting a second set of medical scans from a medical scan database;   generating a second image-to-prompt mapping by selecting one of the set of user interface features to be displayed with each of the medical scans in the second set of medical scans, wherein the second image-to-prompt mapping is different than the first image-to-prompt mapping;   transmitting the second set of medical scans and the set of user interface features to a second client device corresponding to a user selected from a plurality of users, wherein the second client device is not included in the first set of client devices;   receiving a second set of responses from the second client device, wherein the second set of responses is generated by the second client device in response to utilizing the second image-to-prompt mapping to sequentially display each of the second set of medical scans in conjunction with a user interface based on the mapped one of the set of user interface features on a second user interface on a second display device associated with second client device; and   generating second response score data for each response of each of the second set of responses by comparing each of the second set of responses to the truth annotation data of a corresponding medical scan indicated by the second image-to-prompt mapping;   wherein generating the interface feature score data corresponding to each user interface feature in the set of user interface features is further based on the second response score data.   
     
     
         6 . The medical scan interface feature evaluator system of  claim 5 , wherein the first image-to-prompt mapping is generated based on first performance score data of a first set of users corresponding to the first set of client devices, and wherein the second image-to-prompt mapping is generated based on second performance score data of the user. 
     
     
         7 . The medical scan interface feature evaluator system of  claim 1 , wherein the operations further comprise:
 receiving a new medical scan;   generating a new interface feature mapping for the new medical scan by selecting at least one user interface feature from the set of user interface features based on the ranking of the set of user interface features;   transmitting the new medical scan and the new interface feature mapping to a third client device associated with a user selected from a plurality of users;   receiving third response data from the third client device, wherein the third response data is generated by the third client device in response to utilizing the new interface feature mapping to display the new medical scan in conjunction with the selected at least one user interface feature via a third user interface on a third display device associated with the one of the third client device; and   generating new annotation data based on the third response data and mapping the new annotation data to the new medical scan in a medical scan database.   
     
     
         8 . The medical scan interface feature evaluator system of  claim 7 , wherein the at least one user interface feature is selected based by determining at least one highest ranked user interface feature in the ranking. 
     
     
         9 . The medical scan interface feature evaluator system of  claim 7 , wherein the first set of medical scans each correspond to one of a set of scan categories, wherein the ranking includes a set of categorized rankings corresponding to the set of scan categories, and wherein the at least one user interface feature is selected for the new interface feature mapping based on a one of the set of categorized rankings that corresponds to a scan category of the new medical scan. 
     
     
         10 . The medical scan interface feature evaluator system of  claim 1 , wherein a lowest ranked user interface feature in the ranking is automatically removed from the set of user interface features. 
     
     
         11 . A method comprising:
 determining a first image-to-prompt mapping based on one of a set of user interface features to be displayed with each medical scan of a first set of medical scans;   transmitting the first set of medical scans and the first image-to-prompt mapping to a first set of client devices;   receiving a first set of responses from each of the first set of client devices, wherein each first set of responses is generated based on a corresponding one of the first set of client devices displaying at least one of the first set of medical scans in conjunction with a mapped one of the set of user interface features indicated in the first image-to-prompt mapping;   generating first response score data for each response of the first set of responses received from each of the first set of client devices by comparing each response to truth annotation data of a corresponding medical scan of the first set of medical scans indicated by the first image-to-prompt mapping;   generating interface feature score data corresponding to each user interface feature in the set of user interface features based on corresponding first response score data; and   generating a ranking of the set of user interface features based on the interface feature score data.   
     
     
         12 . The method of  claim 11 , wherein at least one response in the first set of responses indicates that a corresponding medical scan is normal in response to one of the first set of client devices displaying one of the set of user interface features that includes a prompt to select one of: that a displayed medical scan is normal or that the displayed medical scan includes an abnormality, and wherein generating the first response score data for the at least one response includes assigning a low score to the one response based on determining that truth annotation data of an at least one corresponding medical scan indicates at least one abnormality. 
     
     
         13 . The method of  claim 11 , wherein at least one response in the first set of responses includes abnormality classification data generated by one of the first set of client devices in response to displaying one of the set of user interface features that includes a prompt to classify an abnormality, and wherein the generating the first response score data of the at least one response includes comparing the abnormality classification data of the at least one response to truth annotation data of an at least one corresponding medical scan. 
     
     
         14 . The method of  claim 11 , wherein at least one response in the first set of responses includes an identified region generated by one of the first set of client devices in response to displaying one of the set of user interface features that includes a prompt to identify a region of the displayed medical scan that includes an abnormality, and wherein the generating the first response score data includes comparing the identified region of the at least one response to truth annotation data of an at least one corresponding medical scan. 
     
     
         15 . The method of  claim 11 , further comprising:
 selecting a second set of medical scans from a medical scan database;   generating a second image-to-prompt mapping by selecting one of the set of user interface features to be displayed with each of the medical scans in the second set of medical scans, wherein the second image-to-prompt mapping is different than the first image-to-prompt mapping;   transmitting the second set of medical scans and the set of user interface features to a second client device corresponding to a user selected from a plurality of users, wherein the second client device is not included in the first set of client devices;   receiving a second set of responses from the second client device, wherein the second set of responses is generated by the second client device in response to utilizing the second image-to-prompt mapping to sequentially display each of the second set of medical scans in conjunction with a user interface based on the mapped one of the set of user interface features on a second user interface on a second display device associated with second client device; and   generating second response score data for each response of each of the second set of responses by comparing each of the second set of responses to truth annotation data of a corresponding medical scan indicated by the second image-to-prompt mapping;   wherein generating the interface feature score data corresponding to each user interface feature in the set of user interface features is further based on the second response score data.   
     
     
         16 . The method of  claim 15 , wherein the first image-to-prompt mapping is generated based on first performance score data of a set of users corresponding to the first set of client devices, and wherein the second image-to-prompt mapping is generated based on second performance score data of the user. 
     
     
         17 . The method of  claim 11 , further comprising;
 receiving a new medical scan;   generating a new interface feature mapping for the new medical scan by selecting at least one user interface feature from the set of user interface features based on the ranking of the set of user interface features;   transmitting the new medical scan and the new interface feature mapping to a third client device associated with a user selected from a plurality of users;   receiving third response data from the third client device, wherein the third response data is generated by the third client device in response to utilizing the new interface feature mapping to display the new medical scan in conjunction with the selected at least one user interface feature via a third user interface on a third display device associated with the one of the third client device; and   generating new annotation data based on the third response data and mapping the new annotation data to the new medical scan in a medical scan database.   
     
     
         18 . The method of  claim 17 , wherein the first set of medical scans each correspond to one of a set of scan categories, wherein the ranking includes a set of categorized rankings corresponding to the set of scan categories, and wherein the at least one user interface feature is selected for the new interface feature mapping based on a one of the set of categorized rankings that corresponds to a scan category of the new medical scan. 
     
     
         19 . The method of  claim 11 , wherein a lowest ranked user interface feature in the ranking is automatically removed from a database of user interface features. 
     
     
         20 . An article of manufacture that includes a non-transitory tangible storage medium that stores operational instructions, that when executed by a processor, causes the processor to:
 determine a first plurality of image-to-prompt mappings based on, for each image-to-prompt mapping in the first plurality of image-to-prompt mappings, at least one of a set of user interface features to be displayed with each medical scan of a first set of medical scans;   transmit the first set of medical scans and a corresponding one of the first plurality of image-to-prompt mappings to each of a first set of client devices;   receive a first set of responses from each of the first set of client devices, wherein each set of responses is generated based on a corresponding one of the first set of client devices displaying each of the first set of medical scans in conjunction with a mapped at least one of the set of user interface features indicated in the corresponding one of the first plurality of image-to-prompt mappings;   generate first response score data for each response of the first set of responses received from each of the first set of client devices by comparing each response to truth annotation data of a corresponding medical scan of the first set of medical scans indicated by corresponding one of the first plurality of image-to-prompt mappings;   generate interface feature score data corresponding to each user interface feature in the set of user interface features based on aggregating corresponding first response score data; and   generate a ranking of the set of user interface features based on the interface feature score data.