Method and system for reconciling meta-data in a data warehouse

Abstract

A data warehouse system comprising a warehouse database and a data warehouse manager communicatively coupled to the database. The warehouse manager is to store data to and retrieve data from the warehouse database and perform metadata reconciliation to update metadata of objects in the data warehouse to reflect a change in metadata of a first object in the warehouse by identifying a first set of objects impacted by the change to the metadata of the first object, identifying a second set of objects on which the first object depends, identifying at least one task that has to be completed to propagate the impact of the change in the metadata of the first object to objects in the first and second sets of objects, performing the task, and dynamically computing additional tasks based on whether or not definitions of objects in the first or second sets of objects changes based upon completion of the at least one task.

What is claimed is: 
     
         1 . A method comprising:
 receiving, at one or more computer systems, a change in a definition of a first object describing a first portion of data stored in a data warehouse;   determining, with one or more processors associated with the one or more computer systems, at least one task to propagate the change in the definition of the first object to metadata of the first object;   reconciling, with one or more processors associated with one or more computer systems, lineage of the first object to identify a set of objects that are impacted by the change to the definition of the first object, each object in the second set of objects describing a second portion of the data stored in the data warehouse different from the first portion of the data;   determining, with the one or more processors associated with the one or more computer systems, at least one additional task to propagate an impact of the change to metadata of one or more objects in the set of objects that are impacted by the change to the definition of the first object; and   rendering, with the one or more processors associated with the one or more computer systems, a visual representation of the impact of the change to the metadata of the one or more objects within a graphical user interface.   
     
     
         2 . The method of  claim 1  further comprising generating a script of all tasks needed to propagate the impact of the change to the metadata of the one or more objects. 
     
     
         3 . The method of  claim 2  further comprising executing the script in response to a command provided by a user via the user interface. 
     
     
         4 . The method of  claim 2  wherein reconciling, with the one or more processors associated with one or more computer systems, the lineage of the first object comprises generating a graph based on relationships between the first object and each object in the set of objects. 
     
     
         5 . The method of  claim 4  wherein rendering, with the one or more processors associated with the one or more computer systems, the visual representation of the impact of the change to the metadata of the one or more objects within the graphical user interface comprises rendering the graph depicting the relationships between the first object and each object in the set of objects. 
     
     
         6 . The method of  claim 1  wherein reconciling, with the one or more processors associated with one or more computer systems, the lineage of the first object comprises performing a lineage analysis within an impact analysis. 
     
     
         7 . The method of  claim 1  further comprising generating updated versions of the first object and each object in the set of objects based upon completion of a set of tasks. 
     
     
         8 . The method of  claim 1  wherein reconciling, with the one or more processors associated with one or more computer systems, the lineage of the first object to identify the set of objects comprises managing hierarchical dependencies among objects based on dependency documents that conform to a common dependency format. 
     
     
         9 . The method of  claim 1  further comprising rendering a visual representation of any determined tasks within the graphical user interface. 
     
     
         10 . The method of  claim 1  further comprising rendering one or more suggestions within the graphical user interface based on any determined tasks. 
     
     
         11 . A non-transitory computer-readable medium storing a computer program product which when executed by a processor of a computer system cause the processor to:
 receive a change in a definition of a first object describing a first portion of data stored in a data warehouse;   determine at least one task to propagate the change in the definition of the first object to metadata of the first object;   reconcile lineage of the first object to identify a set of objects that are impacted by the change to the definition of the first object, each object in the second set of objects describing a second portion of the data stored in the data warehouse different from the first portion of the data;   determine at least one additional task to propagate an impact of the change to metadata of one or more objects in the set of objects that are impacted by the change to the definition of the first object; and   render a visual representation of the impact of the change to the metadata of the one or more objects within a graphical user interface.   
     
     
         12 . The non-transitory computer-readable medium of  claim 11  wherein the computer program product further causes the processor to generate a script of all tasks needed to propagate the impact of the change to the metadata of the one or more objects. 
     
     
         13 . The non-transitory computer-readable medium of  claim 12  wherein the computer program product further causes the processor to execute the script in response to a command provided by a user via the user interface. 
     
     
         14 . The non-transitory computer-readable medium of  claim 12  wherein to reconcile the lineage of the first object the computer program product further causes the processor to generate a graph based on relationships between the first object and each object in the set of objects. 
     
     
         15 . The non-transitory computer-readable medium of  claim 14  wherein to render the visual representation of the impact of the change to the metadata of the one or more objects within the graphical user interface the computer program product further causes the processor to render the graph depicting the relationships between the first object and each object in the set of objects. 
     
     
         16 . The non-transitory computer-readable medium of  claim 11  wherein to reconcile the lineage of the first object the computer program product causes the processor to performing a lineage analysis within an impact analysis. 
     
     
         17 . The non-transitory computer-readable medium of  claim 11  wherein the computer program product further causes the processor to generate updated versions of the first object and each object in the set of objects based upon completion of a set of tasks. 
     
     
         18 . The non-transitory computer-readable medium of  claim 11  wherein to reconcile the lineage of the first object to identify the set of objects the computer program product causes the processor to managing hierarchical dependencies among objects based on dependency documents that conform to a common dependency format. 
     
     
         19 . The method of  claim 1  further comprising rendering one or more suggestions within the graphical user interface based on any determined tasks. 
     
     
         20 . A system comprising:
 a hardware processor; and   a memory storing a set of instructions which when executed by the processor cause the processor to:
 receive a change in a definition of a first object describing a first portion of data stored in a data warehouse; 
 determine at least one task to propagate the change in the definition of the first object to metadata of the first object; 
 reconcile lineage of the first object to identify a set of objects that are impacted by the change to the definition of the first object, each object in the second set of objects describing a second portion of the data stored in the data warehouse different from the first portion of the data; 
 determine at least one additional task to propagate an impact of the change to metadata of one or more objects in the set of objects that are impacted by the change to the definition of the first object; and 
 render a visual representation of the impact of the change to the metadata of the one or more objects within a graphical user interface.