High validity real-world evidence study with deep phenotyping

Abstract

Systems and methods are described for implementing an advanced, “research-grade” or “regulatory-grade,” real-world evidence (RWE) approach. The advanced RWE is able to extract a deep phenotype from rich data sources using advanced technologies including artificial intelligence. The rich data sources include both unstructured data and structured data from electric health records and may include additional data sources such as claims or registries. Systems and methods are also described for validating the deep phenotype which can then be used to create a patient cohort that may be linked to exposure or outcome data to make credible clinical assertions.

1 . A computer-implemented method, comprising:
 extracting, from unstructured data in an electronic health record (EHR), a plurality of clinical concepts using a semantic processing technique;   mapping each of the extracted clinical concepts to a coded clinical concept;   comparing the mapped clinical concepts and concept attributes to inclusion or exclusion criteria to define a cohort of patients within the EHR that satisfy a desired phenotype;   creating a generated gold standard for a portion of the clinical concepts within a portion of the patients within the cohort; and   measuring an accuracy of the semantic processing-based extraction of the clinical concepts for the cohort to determine validity of the cohort with respect to the generated gold standard for a subset of the cohort, based on at least a portion of the inclusion or exclusion criteria.   
     
     
         2 . The method of  claim 1 , wherein the extracting the plurality of clinical concepts comprises:
 (a) obtaining a table of association that maintains associations of clinical concepts;   (b) extracting, using an artificial intelligence technology, from a patient record, the plurality of clinical concepts;   (c) determining a level of support for each extracted clinical concept at least based on an association between the extracted clinical concept and other clinical concepts extracted from the patient record according to the table of association;   (d) identifying, from the plurality of clinical concepts by checking the table of association, a clinical concept representing a symptom already explained by another clinical concept in the plurality of clinical concepts representing a disease; and   (e) filtering the extracted clinical concepts by exclusion of (1) extracted clinical concepts having relatively lower levels of support among the extracted clinical concepts, and (2) the clinical concept identified in (d).   
     
     
         3 . The method of  claim 2 , further comprising:
 constructing the table of association based on a corpus of clinical narratives or medical literature.   
     
     
         4 . The method of  claim 1 , wherein the inclusion or exclusion criteria are generated by at least:
 associating at least a subset of the clinical concepts and concept attributes with the desired phenotype, wherein the desired phenotype satisfies a threshold phenotypic similarity to a phenotype in a randomized controlled trial.   
     
     
         5 . The method of  claim 1 , wherein the inclusion or exclusion criteria are generated by at least:
 associating at least a subset of the clinical concepts and concept attributes with the desired phenotype, wherein the desired phenotype satisfies a threshold phenotypic similarity to a phenotype in an existing or anticipated regulatory-approved label.   
     
     
         6 . The method of  claim 1 , further comprising:
 in response to the accuracy being above a threshold, obtaining, for the cohort, exposure data and outcome data relating to at least a portion of the patients within the cohort of patients; and   implementing a real-world evidence (RWE) study based on the patient phenotype associated with the exposure data or the outcome data for at least one of the patients.   
     
     
         7 . The method of  claim 6 , wherein the implementing of the RWE study comprises comparing outcomes from the outcome data of the cohort with outcomes from an interventional study so that the cohort functions as a synthetic control arm. 
     
     
         8 . The method of  claim 6 , wherein the implementing of the RWE study comprises comparing outcomes of the cohort with outcomes from another cohort or another study to determine comparative effectiveness of at least two treatments. 
     
     
         9 . The method of  claim 6 , wherein the implementing of the RWE study comprises conducting, based on the cohort, an observational study. 
     
     
         10 . The method of  claim 6 , wherein the implementing of the RWE study comprises comparing outcomes of cohorts based on demographically distinct subpopulations on similar treatment regimens to understand heterogeneity of treatment effects on those subpopulations. 
     
     
         11 . The method of  claim 6 , wherein the implementing of the RWE study comprises multiple subgroups to determine preferred design of a randomized controlled trial (RCT). 
     
     
         12 . The method of  claim 6 , wherein the implementing of the RWE study comprises implementing the association of the patient phenotype with the exposure data or the outcome data through data linkage with another data set. 
     
     
         13 . The method of  claim 6 , wherein the implementing of the RWE study comprises implementing the association of the patient phenotype with the exposure data and the outcome data for identifying patient safety events for pharmacovigilance. 
     
     
         14 . A non-transitory computer-readable storage medium configured with instructions executable by one or more processors to cause the one or more processors to perform operations comprising:
 extracting, from unstructured data in an electronic health record (EHR), a plurality of clinical concepts using a semantic processing technique;   mapping each of the extracted clinical concepts to a coded clinical concept;   comparing the mapped clinical concepts and concept attributes to inclusion or exclusion criteria to define a cohort of patients within the EHR that satisfy a desired phenotype;   creating a generated gold standard for a portion of the clinical concepts within a portion of the patients within the cohort; and   measuring an accuracy of the semantic processing-based extraction of the clinical concepts for the cohort to determine validity of the cohort with respect to the generated gold standard for a subset of the cohort, based on at least a portion of the inclusion or exclusion criteria.   
     
     
         15 . The non-transitory computer-readable storage medium of  claim 14 , wherein the extracting the plurality of clinical concepts comprises:
 (a) obtaining a table of association that maintains associations of clinical concepts;   (b) extracting, using an artificial intelligence technology, from a patient record, the plurality of clinical concepts;   (c) determining a level of support for each extracted clinical concept at least based on an association between the extracted clinical concept and other clinical concepts extracted from the patient record according to the table of association;   (d) identifying, from the plurality of clinical concepts by checking the table of association, a clinical concept representing a symptom already explained by another clinical concept in the plurality of clinical concepts representing a disease; and   (e) filtering the extracted clinical concepts by exclusion of (1) extracted clinical concepts having relatively lower levels of support among the extracted clinical concepts, and (2) the clinical concept identified in (d).   
     
     
         16 . The non-transitory computer-readable storage medium of  claim 15 , wherein the operations further comprise:
 constructing the table of association based on a corpus of clinical narratives or medical literature.   
     
     
         17 . The non-transitory computer-readable storage medium of  claim 14 , wherein the inclusion or exclusion criteria are generated by at least:
 associating at least a subset of the clinical concepts and concept attributes with the desired phenotype, wherein the desired phenotype satisfies a threshold phenotypic similarity to a phenotype in a randomized controlled trial.   
     
     
         18 . The non-transitory computer-readable storage medium of  claim 14 , wherein the inclusion or exclusion criteria are generated by at least:
 associating at least a subset of the clinical concepts and concept attributes with the desired phenotype, wherein the desired phenotype satisfies a threshold phenotypic similarity to a phenotype in an existing or anticipated regulatory-approved label.   
     
     
         19 . The non-transitory computer-readable storage medium of  claim 14 , wherein the operations further comprise:
 in response to the accuracy being above a threshold, obtaining, for the cohort, exposure data and outcome data relating to at least a portion of the patients within the cohort of patients; and   implementing a real-world evidence (RWE) study based on the patient phenotype associated with the exposure data or the outcome data for at least one of the patients.   
     
     
         20 . The non-transitory computer-readable storage medium of  claim 19 , wherein the implementing of the RWE study comprises conducting, based on the cohort, an observational study.