Using views of subsets of nodes of a schema to generate data transformation jobs to transform input files in first data formats to output files in second data formats

Abstract

Provided is a method for processing input data in a storage system and in communication with a repository. Views are generated that comprise a tree of nodes selected from a subset of nodes in a hierarchical representation of a schema. The views are saved to the repository. At least one of the views are used to create a job comprising a sequence of data transformation steps to transform the input data described by input schemas to the output data described by output schemas.

What is claimed is: 
     
         1 . A method for processing input data in a storage system, comprising:
 generating views in a computer readable storage medium, each view comprising a tree of nodes selected from a subset of nodes in a hierarchical representation of a schema;   saving the views to a repository comprising computer storage; and   using at least one of the views to create a job comprising a sequence of data transformation steps to transform the input data described by input schemas to output data described by output schemas.   
     
     
         2 . The method of  claim 1 , wherein creating the job comprises:
 receiving a mapping of the nodes in the input schemas to nodes in at least one of the views; and   using the mapping to map the data from the inputs in a first data format into the nodes of the at least one view, wherein the nodes of the at least one view are mapped to output data in a second data format comprising a structured document defining elements corresponding to the subset of nodes in the selected at least one view.   
     
     
         3 . The method of  claim 2 , wherein the first data format comprises a flat text file having text delimited data or a database table, and wherein the second data format comprises an Extended Markup Language (XML) file format. 
     
     
         4 . The method of  claim 1 , wherein the views can be reused in multiple data transformation jobs. 
     
     
         5 . The method of  claim 1 , wherein the data transformation steps use different views from the same input schema to map nodes and data from the inputs to nodes defined by the views. 
     
     
         6 . The method of  claim 1 , wherein the views are used to parse the input data and generate a hierarchical representation of the data described by the views. 
     
     
         7 . The method of  claim 1 , wherein the data transformation steps comprise composing steps using different views from the input schema. 
     
     
         8 . The method of  claim 7 , wherein the transformation steps include join operations on nodes of data from the inputs and map the results of the join operations to the views, and wherein the composing steps merges the results of the join operations mapped to the views and maps the results to one of the views. 
     
     
         9 . The method of  claim 7 , wherein the one of the views used in the job includes a chunked node, wherein the transformation comprises including all data and nodes in a tree of nodes from the chunked node into a string stored at the chunk node, and wherein the chunked node and the string map to one of the nodes in the output data. 
     
     
         10 . The method of  claim 1 , wherein the operations further comprise:
 generating a graphical user interface to enable the user to select inputs, views, and output data to define the job.