Data distillery for signal detection

Abstract

Computer-implemented methods, systems and products for analytics and discovery of patterns or signals. The method includes a set of operations or steps, including collecting data from a plurality of data sources, the data having a plurality of associated data types, and filtering the collected data based on identifying viable data sources from which the data is collected. The method further includes prioritizing discovery objectives based on analyzing the filtering results, and enriching the filtered collected data from viable data sources according to the prioritized discovery objectives. The method further includes extracting one or more signals from the enriched data using one or more machine learning mechanisms in combination with qualified subject matter expertise input, and graphically displaying the extracted signals in a meaningful way to a human operator such that the human operator is enabled to understand importance of extracted signals.

What is claimed is: 
     
       1. A computer implemented system comprising:
 at least one programmable processor; 
 a non-transitory machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to perform operations comprising: 
 receiving information framing a discovery objective regarding aspects of a plan; 
 receiving information identifying data sources deemed to be relevant to the discovery objective; 
 identifying other data sources based on at least one of the received information identifying the data sources deemed to be relevant to the discovery objective and the received information framing the discovery objective; 
 retrieving data based on the information framing the discovery objective from at least some of the data sources deemed to be relevant to the discovery objective by one or more users and the identified other data sources; 
 assessing quality of the data sources from which data was retrieved by determining relevance of data retrieved from the data sources to the discovery objective across a plurality of dimensions including at least two or more of data integration, data resolution, data panorama, data accuracy, or data accessibility; 
 calculating quality indicators indicative of the assessed quality of the data sources; and 
 providing, over a network to one or more participants, the calculated quality indicators indicative of the assessed quality of the data sources from which data was retrieved. 
 
     
     
       2. The system of  claim 1 , wherein the operations further comprise:
 receiving information referencing at least some of one or more data sources for which the quality indicators were calculated; 
 receiving one or more datasets of the data retrieved from at least some of the data sources deemed to be relevant to the discovery objective by the one or more users or from the identified other data sources; and 
 wrangling the one or more datasets into a form that is computationally actionable based on detecting relationships between a plurality of entities or events identified in the data retrieved from one or more of the data sources meeting a certain assessed quality, wherein the strength between at least two of the plurality of entities or events is determined based on a measure associated with how much occurrence of a first event or entity influences the probability of occurrence of a second event or entity. 
 
     
     
       3. The system of  claim 2 , wherein the operations further comprise:
 enriching at least some data from the one or more datasets to generate an enriched form of at least some data corresponding to the one or more datasets, the enriched form being computationally actionable by a user. 
 
     
     
       4. The system of  claim 3 , wherein the operations further comprise:
 extracting one or more signals from the enriched data using one or more machine learning mechanisms in combination with qualified subject matter expertise input, enriching at least some data from the one or more datasets comprising combining one or more data elements from the one or more datasets to create characteristics and variables that make the one or more extracted signals more explicit. 
 
     
     
       5. The system of  claim 3 , wherein the operations further comprise:
 processing the one or more datasets and the enriched form of at least some data to identify one or more of relationships, anomalies and patterns within the one or more datasets. 
 
     
     
       6. The system of  claim 3 , wherein the operations further comprise:
 extracting one or more signals from the enriched data using one or more machine learning mechanisms in combination with qualified subject matter expertise input, enriching at least some data from the one or more datasets comprising combining one or more data elements from the one or more datasets to create characteristics and variables that make the one or more extracted signals more explicit. 
 
     
     
       7. The system of  claim 3 , wherein the operations further comprise:
 processing the one or more datasets and the enriched form of at least some data to identify one or more of relationships, anomalies and patterns within the one or more datasets. 
 
     
     
       8. The system of  claim 2 , wherein the operations further comprise:
 enriching at least some data from the one or more datasets to generate an enriched form of at least some data corresponding to the one or more datasets, the enriched form being computationally actionable by a user. 
 
     
     
       9. The system of  claim 1 , wherein the operations further comprise:
 receiving information referencing at least some of one or more data sources for which the quality indicators were calculated; 
 receiving one or more datasets of the data retrieved from at least some of the data sources deemed to be relevant to the discovery objective by the one or more users or from the identified other data sources; and 
 wrangling the one or more datasets into a form that is computationally actionable based on detecting relationships between a plurality of entities or events identified in the data retrieved from one or more of the data sources meeting a certain assessed quality, wherein the strength between at least two of the plurality of entities or events is determined based on a measure associated with how much occurrence of a first event or entity influences the probability of occurrence of a second event or entity. 
 
     
     
       10. A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
 receiving information framing a discovery objective regarding aspects of a plan; 
 receiving information identifying data sources deemed to be relevant to the discovery objective; 
 identifying other data sources based on at least one of the received information identifying the data sources deemed to be relevant to the discovery objective and the received information framing the discovery objective; 
 retrieving data based on the information framing the discovery objective from at least some of the data sources deemed to be relevant to the discovery objective by one or more users and the identified other data sources; 
 assessing quality of the data sources from which data was retrieved by determining relevance of data retrieved from the data sources to the discovery objective across a plurality of dimensions including at least two or more of data integration, data resolution, data panorama, data accuracy, or data accessibility; 
 calculating quality indicators indicative of the assessed quality of the data sources; and 
 providing, over a network to one or more participants, the calculated quality indicators indicative of the assessed quality of the data sources from which data was retrieved. 
 
     
     
       11. The computer program product of  claim 6 , wherein the operations further comprise:
 receiving information referencing at least some of one or more data sources for which the quality indicators were calculated; 
 receiving one or more datasets of the data retrieved from at least some of the data sources deemed to be relevant to the discovery objective by the one or more users or from the identified other data sources; and 
 wrangling the one or more datasets into a form that is computationally actionable based on detecting relationships between a plurality of entities or events identified in the data retrieved from one or more of the data sources meeting a certain assessed quality, wherein the strength between at least two of the plurality of entities or events is determined based on a measure associated with how much occurrence of a first event or entity influences the probability of occurrence of a second event or entity. 
 
     
     
       12. The computer program product of  claim 11 , wherein the operations further comprise:
 enriching at least some data from the one or more datasets to generate an enriched form of at least some data corresponding to the one or more datasets, the enriched form being computationally actionable by a user. 
 
     
     
       13. The computer program product of  claim 12 , wherein the operations further comprise:
 extracting one or more signals from the enriched data using one or more machine learning mechanisms in combination with qualified subject matter expertise input, wherein enriching at least some data from the one or more datasets comprises combining one or more data elements from the one or more datasets to create characteristics and variables that make the one or more signals more explicit. 
 
     
     
       14. The computer program product of  claim 12 , wherein the operations further comprise:
 processing the one or more datasets and the enriched form of at least some data to identify one or more of relationships, anomalies and patterns within the one or more datasets. 
 
     
     
       15. A computer-implemented method comprising:
 receiving information framing a discovery objective regarding aspects of a plan; 
 receiving information identifying data sources deemed to be relevant to the discovery objective; 
 identifying other data sources based on at least one of the received information identifying the data sources deemed to be relevant to the discovery objective and the received information framing the discovery objective; 
 retrieving data based on the information framing the discovery objective from at least some of the data sources deemed to be relevant to the discovery objective by one or more users and the identified other data sources; 
 assessing quality of the data sources from which data was retrieved by determining relevance of data retrieved from the data sources to the discovery objective across a plurality of dimensions including at least two or more of data integration, data resolution, data panorama, data accuracy, or data accessibility; 
 calculating quality indicators indicative of the assessed quality of the data sources; and 
 providing, over a network to one or more participants, the calculated quality indicators indicative of the assessed quality of the data sources from which data was retrieved.