Automated soap note evaluation using machine learning models

Abstract

Techniques are disclosed for automatically evaluating SOAP notes. A method comprises accessing a Subjective, Objective, Assessment and Plan (SOAP) note and a checklist that includes checklist facts; using a first machine-learning model prompt to extract SOAP note facts from the SOAP note; using one or more second machine-learning model prompts to generate feedback for the SOAP note, the feedback indicating whether individual checklist facts are supported by at least one of the SOAP note facts, and whether individual SOAP note facts are supported by at least one of the checklist facts; and generating a score for the SOAP note based on the feedback.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 accessing a Subjective, Objective, Assessment and Plan (SOAP) note and a checklist that includes checklist facts;   using a first machine-learning model prompt to extract SOAP note facts from the SOAP note;   using one or more second machine-learning model prompts to generate feedback for the SOAP note, the feedback indicating whether individual checklist facts are supported by at least one of the SOAP note facts, and whether individual SOAP note facts are supported by at least one of the checklist facts; and   generating a score for the SOAP note based on the feedback.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using a first prompt of the one or more second machine-learning model prompts to determine whether each checklist fact is included the SOAP note facts. 
     
     
         3 . The computer-implemented method of  claim 2 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using the first prompt to determine an importance level of each of the checklist facts. 
     
     
         4 . The computer-implemented method of  claim 2 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using a second prompt to determine whether each SOAP note fact corresponds to at least one checklist fact in the checklist facts. 
     
     
         5 . The computer-implemented method of  claim 1 , wherein the SOAP note is generated using one or more machine learning models based at least in-part on a text transcript corresponding to an interaction between a healthcare provider and a patient of the healthcare provider. 
     
     
         6 . The computer-implemented method of  claim 5 , wherein the checklist facts are extracted from the text transcript, and wherein each of the checklist facts is expressed as a sentence. 
     
     
         7 . The computer-implemented method of  claim 1 , wherein each SOAP note fact in the SOAP note facts is an atomic sentence, and wherein using the first machine-learning model prompt to extract the SOAP note facts comprises using the first machine-learning model prompt to generate atomic sentences from the SOAP note. 
     
     
         8 . The computer-implemented method of  claim 1 , wherein the feedback provides an indication of which checklist facts are not included in the SOAP note facts and which SOAP note facts are not included in the checklist facts. 
     
     
         9 . The computer-implemented method of  claim 8 , wherein generating the score for the SOAP note based on the feedback comprises calculating a SOAP note score based on the feedback. 
     
     
         10 . The computer-implemented method of  claim 1 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using a prompt of the one or more second machine-learning model prompts to determine that a SOAP note fact, indicated to support a checklist fact, contradicts the checklist fact. 
     
     
         11 . A system comprising:
 one or more processing systems; and   one or more computer-readable media storing instructions which, when executed by the one or more processing systems, cause the system to perform operations comprising:
 accessing a Subjective, Objective, Assessment and Plan (SOAP) note and a checklist that includes checklist facts; 
 using a first machine-learning model prompt to extract SOAP note facts from the SOAP note; 
 using one or more second machine-learning model prompts to generate feedback for the SOAP note, the feedback indicating whether individual checklist facts are supported by at least one SOAP note fact, and whether individual SOAP note facts are supported by at least one fact in the checklist facts; and 
 generating a score for the SOAP note based on the feedback. 
   
     
     
         12 . The system of  claim 11 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using a first prompt of the one or more second machine-learning model prompts to determine whether each checklist fact is included the SOAP note facts. 
     
     
         13 . The system of  claim 12 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using the first prompt to determine an importance level of each of the checklist facts. 
     
     
         14 . The system of  claim 12 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using a second prompt to determine whether each of the SOAP note facts corresponds to at least one of the checklist facts. 
     
     
         15 . The system of  claim 11 , wherein the SOAP note is generated using one or more machine learning models based at least in-part on a text transcript corresponding to an interaction between a healthcare provider and a patient of the healthcare provider. 
     
     
         16 . The system of  claim 15 , wherein the checklist facts are extracted from the text transcript and wherein each of the checklist facts is expressed as a sentence. 
     
     
         17 . The system of  claim 11 , wherein each fact in the SOAP note facts is an atomic sentence, and wherein using the first machine-learning model prompt to extract the SOAP note facts from the SOAP note comprises using the first machine-learning model prompt to generate atomic sentences from the SOAP note. 
     
     
         18 . The system of  claim 11 , wherein the feedback provides an indication of which checklist facts are not included in the SOAP note facts and which SOAP note facts are not included in the checklist facts. 
     
     
         19 . The system of  claim 18 , wherein generating the score for the SOAP note based on the feedback comprises calculating a SOAP note score based on the feedback. 
     
     
         20 . The system of  claim 11 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using a prompt of the one or more second machine-learning model prompts to determine that a SOAP note fact, indicated to support a checklist fact, contradicts the checklist fact. 
     
     
         21 . One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause a system to perform operations comprising:
 accessing a Subjective, Objective, Assessment and Plan (SOAP) note and a checklist that includes checklist facts;   using a first machine-learning model prompt to extract SOAP note facts from the SOAP note;   using a one or more second machine-learning model prompts to generate feedback for the SOAP note, the feedback indicating whether individual checklist facts are supported by at least one SOAP note fact, and whether individual SOAP note facts are supported by at least one checklist fact; and   generating a score for the SOAP note based on the feedback.   
     
     
         22 . The one or more non-transitory computer-readable media of  claim 21 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using a first prompt of the one or more second machine-learning model prompts to determine whether each of the checklist facts is included the SOAP note facts. 
     
     
         23 . The one or more non-transitory computer-readable media of  claim 22 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using the first prompt to determine an importance level of each of the checklist facts. 
     
     
         24 . The one or more non-transitory computer-readable media of  claim 22 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using a second prompt to determine whether each of the SOAP note facts corresponds to at least one of the checklist facts. 
     
     
         25 . The one or more non-transitory computer-readable media of  claim 21 , wherein the SOAP note is generated using one or more machine learning models based at least in-part on a text transcript corresponding to an interaction between a healthcare provider and a patient of the healthcare provider. 
     
     
         26 . The one or more non-transitory computer-readable media of  claim 25 , wherein the checklist facts are extracted from the text transcript, and wherein each of the checklist facts is expressed as a sentence. 
     
     
         27 . The one or more non-transitory computer-readable media of  claim 21 , wherein each SOAP note fact is an atomic sentence, and wherein using the first machine-learning model prompt to extract the SOAP note facts comprises using the first machine-learning model prompt to generate atomic sentences from the SOAP note. 
     
     
         28 . The one or more non-transitory computer-readable media of  claim 21 , wherein the feedback provides an indication of which of the checklist facts are not included in the SOAP note facts and which of the SOAP note facts are not included in the checklist facts. 
     
     
         29 . The one or more non-transitory computer-readable media of  claim 28 , wherein generating the score for the SOAP note based on the feedback comprises calculating a SOAP note score based on the feedback. 
     
     
         30 . The one or more non-transitory computer-readable media of  claim 21 , wherein using the one or more second machine-learning model prompts to generate the feedback for the SOAP note comprises using a prompt of the one or more second machine-learning model prompts to determine that a SOAP note fact, indicated to support a checklist fact, contradicts the checklist fact.