Multi-hop evidence pursuit for medical decision making

Abstract

Methods and systems include generating a first question relating to supporting an input claim. A search is performed based on the first question to identify evidence relating to the input claim. An answer to the first question is generated based on the evidence. Additional questions are iteratively generated, with searches being performed based on the additional questions, and with answers to the additional questions being generated until a predetermined stop condition is reached. The input claim is classified by predicting a label based on evidence identified by the searches.

What is claimed is: 
     
         1 . A computer-implemented method, comprising:
 generating a first question relating to supporting an input claim;   performing a search based on the first question to identify evidence relating to the input claim;   generating an answer to the first question based on the evidence;   iteratively generating additional questions, performing searches based on the additional questions, and generating answers to the additional questions until a predetermined stop condition is reached; and   classifying the input claim by predicting a label based on evidence identified by the searches.   
     
     
         2 . The method of  claim 1 , wherein generating the first question is performed using a machine learning model that includes a sequence-to-sequence encoder-decoder transformer model. 
     
     
         3 . The method of  claim 2 , wherein generating the additional questions is performed using a large language model. 
     
     
         4 . The method of  claim 1 , wherein generating the additional questions can include an output of “true” or “false” and wherein the stop condition includes determining that a most recent question of the additional question is “true” or “false.” 
     
     
         5 . The method of  claim 1 , further comprising extracting text from a document retrieved by the search as the evidence. 
     
     
         6 . The method of  claim 5 , wherein extracting text from the document includes selecting a window of text that includes more than a threshold percentage of words from a search snippet. 
     
     
         7 . The method of  claim 1 , further comprising paraphrasing the evidence identified by the searches, wherein classifying the input claim includes using a large language model to generate the label using an input that includes the first question, the additional questions, the answers, and the paraphrased evidence. 
     
     
         8 . The method of  claim 1 , wherein generating the answers includes processing the question using a large language model, with the evidence as an input. 
     
     
         9 . The method of  claim 1 , wherein the search is performed over public medical information and over a patient's medical records for use in medical decision making. 
     
     
         10 . The method of  claim 9 , wherein the input claim is a claim relating to a medical condition of the patient, further comprising performing a treatment action responsive to the label. 
     
     
         11 . A system, comprising:
 a hardware processor; and   a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to:
 generate a first question relating to supporting an input claim; 
 perform a search based on the first question to identify evidence relating to the input claim; 
 generate an answer to the first question based on the evidence; 
 iteratively generate additional questions, perform searches based on the additional questions, and generate answers to the additional questions until a predetermined stop condition is reached; and 
 classify the input claim by predicting a label based on evidence identified by the searches. 
   
     
     
         12 . The system of  claim 11 , wherein generation of the first question is performed using a machine learning model that includes a sequence-to-sequence encoder-decoder transformer model. 
     
     
         13 . The system of  claim 12 , wherein generation of the additional questions is performed using a large language model. 
     
     
         14 . The system of  claim 11 , wherein generation of the additional questions can include an output of “true” or “false” and wherein the stop condition includes a determination that a most recent question of the additional question is “true” or “false.” 
     
     
         15 . The system of  claim 11 , wherein the computer program further causes the hardware processor to extract text from a document retrieved by the search as the evidence. 
     
     
         16 . The system of  claim 15 , wherein extraction of text from the document includes selection of a window of text that includes more than a threshold percentage of words from a search snippet. 
     
     
         17 . The system of  claim 11 , wherein the computer program further causes the hardware processor to paraphrase the evidence identified by the searches, wherein classification of the input claim includes using a large language model to generate the label using an input that includes the first question, the additional questions, the answers, and the paraphrased evidence. 
     
     
         18 . The system of  claim 11 , wherein generation of the answers includes processing the question using a large language model, with the evidence as an input. 
     
     
         19 . The system of  claim 11 , wherein the search is performed over public medical information and over a patient's medical records for use in medical decision making. 
     
     
         20 . The system of  claim 19 . wherein the input claim is a claim relating to a medical condition of the patient. wherein the computer program further causes the hardware processor to perform a treatment action responsive to the label.