Component Based Approach to Building Data Integration Tools

Abstract

A method facilitating interoperability across the integrating platforms is provided by composing purpose specific integrating tools using conceptual modeling framework and a set of primitive foundational components. The model driven architecture provides easy migration from one solution architecture to another. Also, the present invention discloses a method to migrate from one implementation to another using set of composed purpose specific integration tools.

1 ) A method for composing integrating tools for facilitating interoperability across integrating platforms, the said method comprising processor implemented steps of:
 modeling plurality of data entities at multiple levels of abstraction in a source schema and capturing relationship between the data entities using a query language in conjugation with path expressions;   mapping between the modeled data entities of the source scheme to obtain a unified model query using a mapping component ;   translating the unified model query of the source schema into an equivalent target queries on a target schema and checking for containment relationship between the queries thereof using a query translation component;   translating one or more target queries into equivalent target data flow graphs using a query to DFG translation component and;   executing the data flow graphs directly or translating the target data flow graphs into third party integrating platform and executing it thereof using a DFG to query translation component.   
     
     
         2 ) The method of  claim 1 , wherein the modeling of data entities at multiple levels is achieved by conceptual modeling framework using an object model or an entity relationship model as a modeling language. 
     
     
         3 ) The method of  claim 1 , wherein the source schema is defined at multiple levels using conceptual model for at least one level and physical level or conceptual model for subsequent levels. 
     
     
         4 ) The method of  claim 1 , wherein the query language and the path expressions captures association relationship between the data entities and gets translated into corresponding structured query language following a translation rule. 
     
     
         5 ) The method of  claim 1 , wherein the path expression are governed by a set of rules to support association relationships; the said rules including:
 selecting from a cartesian product of relations listed in ‘from’ clause of query, the relations that satisfy ‘where’ clause and the path expressions; and projecting out columns specified during selection; wherein the path expressions have implicit existential quantification and;   allowing the path expressions in ‘select’ clause of the query provided *:1 (many to one) associations are involved.   
     
     
         6 ) The method of composing platform specific integrating tools as claimed in  claim 5 , wherein the governing rule allows each association segment of the path expression to introduce two relations in the ‘from’ clause, corresponding to the association and associated class, and corresponding join conditions in the ‘where’ clause of the query. 
     
     
         7 ) The method of  claim 1 , wherein the mapping is specified by either of global-as-view (GAV) or local-as-view (LAV) or GLAV mappings. 
     
     
         8 ) The method of  claim 1 , wherein the target query is translated into an equivalent target data flow graph in steps of:
 segmenting a given query containing subgoals from multiple sources such that each maximally connected subgraph that contains at least one relation node constitutes a query segment;   checking for an already existing subquery having an equivalent query segment for reuse;   factoring the query segments into subquery executable on a single source and;   constructing the target data flow graph from segmented query representation expressing nodes and edges corresponding to subgoals of query.   
     
     
         9 ) The method of composing platform specific integrating tools as claimed in  claim 1 , wherein the data flow graphs are represented by a meta model, the said meta model further defined by data flow graph operators wherein the said operators includes a set of input and output ports along with associated expressions. 
     
     
         10 ) The method of composing platform specific integrating tools as claimed in  claim 1 , wherein the data flow graph (DFG) to query translation component translates the data flow graph consisting of one target relation from one target database and at least one source relation from at least one source database, wherein the translation is performed by:
 formulating an outer query corresponding to target DFG operator in a suitable format and;   formulating sub queries corresponding to each inner operator of the data flow graph in a nested manner and   optimizing the formulated query by removing duplicate sub queries.   
     
     
         11 ) The method of composing platform specific integrating tools as claimed in  claim 10 , wherein the target DFG operator is formulated into its corresponding query using a specified format, the said format comprising
 select clause containing names of output ports;   from clause containing sub queries corresponding to the operators, the said operators having their output port connected to an input port of the operator;   where clause derived from an associated expression of the operator, wherein semantics of the said operator deciding a form of said clause.   
     
     
         12 ) The method of  claim 1 , wherein the composed integrating tools can be either of EII or ETL or data migration or data synchronization tool or a combination thereof. 
     
     
         13 ) The method of  claim 1 , wherein the integrating tool, particularly a data integration tool can be composed in the steps of:
 building a source model and a unified global model using a modeling framework;   mapping between the source model and the global model using a mapping component;   specifying entity matching specifications on the global model to generate object map using a matching component;   translating query updates on the source model into equivalent query updates on matching sources and;   executing the said query updates on the sources using a query translator component.   
     
     
         14 ) The method of  claim 13 , wherein the matching component takes matching specifications expressed on the global model and matches instances of a class across plurality of disparate databases by presenting all matching object pairs from said instances as one or more tuple called object map. 
     
     
         15 ) The method of  claim 1 , wherein the integration tool, particularly a data migration tool is composed in the steps of:
 modeling a source schema and a target schema using a modeling component;   mapping between the source schema and the target schema using a mapping component;   formulating queries corresponding to each target table and translating the said queries into source specific queries using a query translation component and translating the said source specific queries into equivalent data flow graphs using a query to DFG translation component, and executing the said data flow graphs to migrate data from the source schema to the target schema;   
     
     
         16 ) A computer program product comprising a computer usable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for composing integrating tools for facilitating interoperability across integrating platforms, said method comprising:
 modeling plurality of data entities at multiple levels of abstraction in a source schema and capturing relationship between the data entities using a query language in conjugation with path expressions;   mapping between the modeled data entities of the source scheme to obtain a unified model query using a mapping component ;   translating the unified model query of the source schema into an equivalent target queries on a target schema and checking for containment relationship between the queries thereof using a query translation component;   translating one or more target queries into equivalent target data flow graphs using a query to DFG translation component and;   executing the data flow graphs directly or translating the target data flow graphs into third party integrating platform and executing it thereof using a DFG to query translation component.