Systems and methods for interest-driven business intelligence systems including geo-spatial data

Abstract

Systems and methods for interest-driven business intelligence systems including geo-spatial data in accordance with embodiments of the invention are illustrated. An interest-driven business intelligence system including raw data storage and perform extract, transform, and load processes, a data mart, and an intermediate processing layer, wherein the intermediate processing layer is configured to automatically generate metadata describing the raw data, derive reporting data requirements, and compile an interest-driven data pipeline based upon the reporting data requirements, where compiling the interest-driven data pipeline includes generating ETL processing jobs to generate geo-spatial data from the raw data, determining bounding data, bounding the filtered raw data based on the bounding data, generating geo-spatial data, and storing the geo-spatial data, generating reporting data including data satisfying the reporting data requirements based on the geo-spatial data, and storing the reporting data in the data mart for exploration by an interest-driven data visualization system.

What is claimed is: 
     
       1. An interest-driven business intelligence system, comprising:
 raw data storage configured to contain raw data and perform extract, transform, and load (ETL) processes; 
 a data mart configured to contain metadata that describes the raw data; and 
 an intermediate processing layer; 
 wherein the intermediate processing layer is configured to automatically:
 generate metadata describing the raw data; 
 derive reporting data requirements from at least one report specification based on the metadata; and 
 compile an interest-driven data pipeline based upon the reporting data requirements, wherein the compiling of the interest-driven data pipeline comprises:
 generating ETL processing jobs to generate geo-spatial data from the raw data by:
 filtering the raw data based on the metadata describing the raw data; 
 determining bounding data based on the metadata describing the raw data, wherein the bounding data corresponds with one or more coordinates that identify a single point, a set of points, and/or a location within a map, and wherein the bounding data comprises at least one dimension identified in the metadata describing the raw data; 
 bounding the filtered raw data based on the bounding data; 
 generating geo-spatial data based on the bounded filtered raw data, wherein the geo-spatial data includes a set of data; and 
 storing the geo-spatial data in the data mart; mart using a defined bounding area, comprising: 
 aggregating the set of data into bins of data; and 
 utilizing the bins of data to identify pieces of the set of data that fall on the defined bounding area, pieces of the set of data that fall outside the defined bounding area, or both, wherein the defined bounding area corresponds to bounding data describing one or more locations within a space; 
 
 generating reporting data including data satisfying the reporting data requirements based on the geo-spatial data; and 
 storing the reporting data in the data mart for exploration by an interest-driven data visualization system. 
 
 
 
     
     
       2. The interest-driven business intelligence system of  claim 1 , wherein the raw data comprises unstructured data. 
     
     
       3. The interest-driven business intelligence system of  claim 1 , wherein the raw data storage is a data warehouse. 
     
     
       4. The interest-driven business intelligence system of  claim 3 , wherein the data warehouse is implemented utilizing a system selected from the group consisting of a distributed computing system, a database management system, and a NoSQL database. 
     
     
       5. The interest-driven business intelligence system of  claim 3 , wherein the data warehouse is a distributed computing system. 
     
     
       6. The interest-driven business intelligence system of  claim 3 , wherein the data warehouse is configured to store data generated utilizing the intermediate processing layer. 
     
     
       7. The interest-driven business intelligence system of  claim 3 , wherein the intermediate processing layer is configured to generate data warehouse requests. 
     
     
       8. The interest-driven business intelligence system of  claim 7 , wherein the data warehouse requests comprise MapReduce operations. 
     
     
       9. The interest-driven business intelligence system of  claim 1 , wherein the intermediate processing layer is implemented utilizing a system selected from the group consisting of a distributed computing system, a database management system, and a NoSQL database system. 
     
     
       10. The interest-driven business intelligence system of  claim 1 , further comprising an interest-driven data visualization system;
 wherein the interest-driven data visualization system is configured to:
 receive metadata describing the raw data from the intermediate processing layer; and 
 generate a user interface enabling user exploration of the metadata to define at least one report specification, where the user exploration involves selection of additional reporting data based on the metadata. 
 
 
     
     
       11. The interest-driven business intelligence system of  claim 10 , wherein the interest-driven data visualization system is configured to display an indication based upon interactive exploration of the at least one report. 
     
     
       12. The interest-driven business intelligence system of  claim 1 , wherein the data mart is further configured to contain aggregate data, where aggregate data comprises structured data generated using ETL processes from the raw data. 
     
     
       13. The interest-driven business intelligence system of  claim 1 , wherein the data mart is contained within the intermediate processing layer. 
     
     
       14. The interest-driven business intelligence system of  claim 1 , wherein bounding the filtered raw data based on the bounding data comprises determining pieces of raw data that identifying a location that is within the location described by the bounding data. 
     
     
       15. The interest-driven business intelligence system of  claim 14 , wherein:
 bounding the filtered raw data based on the bounding data further comprises mapping at least two dimensions within the filtered raw data to a common set of data; and 
 wherein the common set of data is based on the bounding data. 
 
     
     
       16. The interest-driven business intelligence system of  claim 1 , wherein the bounding data comprises a latitude and a longitude. 
     
     
       17. The interest-driven business intelligence system of  claim 1 , wherein the bounding data comprises a location. 
     
     
       18. The interest-driven business intelligence system of  claim 17 , wherein the bounding data comprises a bounding region described relative to the location. 
     
     
       19. An interest-driven business intelligence system, comprising:
 raw data storage configured to contain raw data and perform extract, transform, and load (ETL) processes; 
 a data mart configured to contain metadata that describes the raw data wherein the data mart is further configured to contain aggregate data, and wherein the aggregate data comprises structured data generated using ETL processes from the raw data; and 
 an intermediate processing layer; 
 wherein the intermediate processing layer is configured to automatically:
 generate metadata describing the raw data; 
 derive reporting data requirements from at least one report specification based on the metadata; and 
 compile an interest-driven data pipeline based upon the reporting data requirements, wherein compiling the interest-driven pipeline further comprises:
 generating ETL processing jobs to generate aggregate data from the raw data by:
 filtering the raw data based on the metadata describing the raw data; 
 applying transformations to the raw data based on the metadata describing the raw data; 
 generating aggregate data based on the transformed data; 
 determining bounding data based on the metadata describing the raw data, wherein the bounding data comprises at least one dimension identified in the metadata describing the raw data; 
 bounding the filtered raw data based on the bounding data; 
 generating geo-spatial data based on the bounded filtered raw data; and 
 storing the aggregate data and the geo-spatial data in the data mart; 
 
 generating reporting data including data satisfying the reporting data requirements based on the geo-spatial data; and 
 storing the reporting data in the data mart for exploration by an interest-driven data visualization system. 
 
 
 
     
     
       20. The interest-driven business intelligence system of  claim 19 , wherein compiling the interest-driven pipeline further comprises generating ETL processing jobs to generate aggregate data from geo-spatial data by:
 identifying at least one dimension within a piece of geo-spatial data; 
 obtaining raw data corresponding to the identified at least one dimension; 
 applying transformations to the obtained raw data based on the metadata describing the obtained raw data; 
 generating aggregate data based on the transformed data; and 
 storing the aggregate data in the data mart. 
 
     
     
       21. The interest-driven business intelligence system of  claim 19 , wherein compiling the interest-driven pipeline further comprises generating ETL processing jobs to generate geo-spatial data from aggregate data by:
 identifying at least one dimension within a piece of aggregate data; 
 obtaining raw data corresponding to the identified at least one dimension; 
 filtering the obtained raw data based on the metadata describing the obtained raw data; 
 determining bounding data based on the metadata describing the obtained raw data; 
 bounding the filtered obtained raw data based on the bounding data; 
 generating geo-spatial data based on the bounded data; and 
 storing the geo-spatial data in the data mart. 
 
     
     
       22. A method for creating a report utilizing an interest-driven business intelligence system, comprising:
 generating, using an intermediate processing layer, metadata describing raw data; 
 deriving, using the intermediate processing layer, reporting data requirements from at least one report specification based on the metadata; and 
 compiling, using the intermediate processing layer, an interest-driven data pipeline based upon the reporting data requirements, wherein the compiling of the interest-driven data pipeline comprises:
 generating extract, transform, and load (ETL) processing jobs to generate geo-spatial data from the raw data by:
 filtering the raw data based on the metadata describing the raw data; 
 determining bounding data based on the metadata describing the raw data, wherein the bounding data corresponds with one or more coordinates that identify a single point, a set of points, and/or a location within a map, and wherein the bounding data comprises at least one dimension identified in the metadata describing the raw data; 
 bounding the filtered raw data based on the bounding data; 
 generating geo-spatial data based on the bounded filtered raw data, wherein the geo-spatial data includes a set of data; and 
 storing the geo-spatial data in a data mart; mart using a defined bounding area, comprising:
 aggregating the set of data into bins of data; and 
 utilizing the bins of data to identify pieces of the set of data that fall on the defined bounding area, pieces of the set of data that fall outside the defined bounding area, or both, wherein the defined bounding area corresponds to bounding data describing one or more locations within a space; 
 
 
 generating reporting data including data satisfying the reporting data requirements based on the geo-spatial data; and 
 storing the reporting data in the data mart for exploration by an interest-driven data visualization system.