Data upload—process integration tool

Abstract

An upload tool includes a network interface, a table generation engine, and an upload engine. The network interface receives a request to generate a data table for a plurality of files. The request includes a username and password for the files, and one or more column definitions. The table generation engine generates the data table that includes the column definitions and a plurality of data rows, each row comprising data from at least one of the plurality of files. The upload engine creates a plurality of batches from the data table and uploads in batch in parallel.

What is claimed is: 
     
       1. An upload tool comprising:
 a network interface configured to:
 receive a request to generate a data table from a plurality of files, the plurality of files are tab delimited files, the request comprising:
 a username and a password for each of the plurality of files; 
 a data table name 
 a data table description; 
 one or more column definitions, each column definition indicating a data type in a column of the data table; and 
 a predetermined number of rows; and 
 
 receive the plurality of files based on the username and the password for each of the plurality of files; 
 
 a table generation engine configured to:
 generate the data table, the data table comprising:
 the data table name; 
 the data table description; 
 one or more columns, each column having the data type indicated by the one or more column definitions one of the one or more column definitions; and 
 a plurality of data rows, each row comprising data from at least one of the plurality of files; and 
 
 generate authentication information for the data table, the authentication information comprising the username, the password, and biometric data, wherein a user submits the username, the password, and the biometric data before the user is permitted to access the data table; 
 
 an upload engine configured to:
 determine whether the data table comprises more than the predetermined number of rows; and 
 upon determination that the data table comprises more than the predetermined number of rows, create a plurality of batches, each batch comprising a subset of the plurality of data rows; and 
 upload each batch of the plurality of batches, wherein uploading each batch of the plurality of batches comprises:
 determining a network node of a plurality of network nodes; 
 instructing the network node to upload the batch; and 
 communicating the batch to the determined network node, wherein the determined network node for a first batch of the plurality of batches is different from the determined network node for a second batch of the plurality of batches, the determined network node for the first batch and the determined network node for the second batch are configured to upload the first and second batches in parallel. 
 
 
 
     
     
       2. The upload tool of  claim 1 , wherein the upload engine is further configured to upload each of the plurality of batches to a previous data table of the server to create a new data table, the new data table comprising data from the previous data table and the data table. 
     
     
       3. The upload tool of  claim 1 , further comprising a search engine configured to:
 generate a SQL search request of the data table, the search comprising at least part of the authentication information; and 
 receive a search result, the search result comprising at least part of the data from the data table in response to the search request. 
 
     
     
       4. The upload tool of  claim 1 , wherein the upload engine is further configured to indicate an upload status comprising one of upload in progress, upload complete, and upload error. 
     
     
       5. The upload tool of  claim 1 , further comprising a viewing engine configured to:
 display the data table; 
 display an editing screen, the editing screen allowing a user to edit the data table; and 
 display a delete option, the delete option allowing a user to delete the data table. 
 
     
     
       6. The upload tool of  claim 1 , wherein the viewing engine is further configured to display an expiration date, the expiration data indicating a date the data table will be deleted. 
     
     
       7. A method comprising:
 receiving a request to generate a data table from a plurality of files, the plurality of files are tab delimited files, the request comprising:
 a username and a password for each of the plurality of files; 
 a data table name 
 a data table description; 
 one or more column definitions, each column definition indicating a data type in a column of the data table; and 
 a predetermined number of rows; 
 
 receiving the plurality of files based on the username and the password for each of the plurality of files; 
 generating the data table, the data table comprising:
 the data table name; 
 the data table description; 
 one or more columns, each column having the data type indicated by the one or more column definitions one of the one or more column definitions; and 
 a plurality of data rows, each row comprising data from at least one of the plurality of files; 
 
 generating authentication information for the data table, the authentication information comprising the username, the password, and biometric data, wherein a user submits the username, the password, and the biometric data before the user is permitted to access the data table; 
 determining whether the data table comprises more than the predetermined number of rows; and 
 upon determination that the data table comprises more than the predetermined number of rows, creating a plurality of batches, each batch comprising a subset of the plurality of data rows; and 
 uploading each batch of the plurality of batches, wherein uploading each batch of the plurality of batches comprises:
 determining a network node of a plurality of network nodes; 
 instructing the network node to upload the batch; and 
 communicating the batch to the determined network node, wherein the determined network node for a first batch of the plurality of batches is different from the determined network node for a second batch of the plurality of batches, the determined network node for the first batch and the determined network node for the second batch are configured to upload the first and second batches in parallel. 
 
 
     
     
       8. The method of  claim 7 , further comprising uploading each of the plurality of batches to a previous data table of the server to create a new data table, the new data table comprising data from the previous data table and the data table. 
     
     
       9. The method of  claim 7 , further comprising:
 generating a SQL search request of the data table, the search comprising at least part of the authentication information; and 
 receiving a search result, the search result comprising at least part of the data from the data table in response to the search request. 
 
     
     
       10. The method of  claim 7 , further comprising indicating an upload status comprising one of upload in progress, upload complete, and upload error. 
     
     
       11. The method of  claim 7 , further comprising:
 displaying the data table; 
 displaying an editing screen, the editing screen allowing a user to edit the data table; and 
 displaying a delete option, the delete option allowing a user to delete the data table. 
 
     
     
       12. The method of  claim 7 , further comprising displaying an expiration date, the expiration data indicating a date the data table will be deleted. 
     
     
       13. A system comprising:
 a user device; and 
 an upload tool configured to:
 receive a request to generate a data table from a plurality of files, the plurality of files are tab delimited files, the request comprising:
 a username and a password for each of the plurality of files; 
 a data table name 
 a data table description; 
 one or more column definitions, each column definition indicating a data type in a column of the data table; and 
 a predetermined number of rows; 
 
 receive the plurality of files based on the username and the password for each of the plurality of files; 
 generate the data table, the data table comprising:
 the data table name; 
 the data table description; 
 one or more columns, each column having the data type indicated by the one or more column definitions one of the one or more column definitions; and 
 a plurality of data rows, each row comprising data from at least one of the plurality of files; 
 
 generate authentication information for the data table, the authentication information comprising the username, the password, and biometric data, wherein a user submits the username, the password, and the biometric data before the user is permitted to access the data table; 
 determine whether the data table comprises more than the predetermined number of rows; and 
 upon determination that the data table comprises more than the predetermined number of rows, create a plurality of batches, each batch comprising a subset of the plurality of data rows; and 
 upload each batch of the plurality of batches, wherein uploading each batch of the plurality of batches comprises:
 determining a network node of a plurality of network nodes; 
 instructing the network node to upload the batch; and 
 communicating the batch to the determined network node, wherein the determined network node for a first batch of the plurality of batches is different from the determined network node for a second batch of the plurality of batches, the determined network node for the first batch and the determined network node for the second batch are configured to upload the first and second batches in parallel. 
 
 
 
     
     
       14. The system of  claim 13 , wherein the upload tool is further configured to upload each of the plurality of batches to a previous data table of the server to create a new data table, the new data table comprising data from the previous data table and the data table. 
     
     
       15. The system of  claim 13 , wherein the upload tool is further configured to:
 generate a SQL search request of the data table, the search comprising at least part of the authentication information; and 
 receive a search result, the search result comprising at least part of the data from the data table in response to the search request. 
 
     
     
       16. The system of  claim 13 , wherein the upload tool is further configured to indicate an upload status comprising one of upload in progress, upload complete, and upload error. 
     
     
       17. The system of  claim 13 , wherein the upload tool is further configured to:
 display the data table; 
 display an editing screen, the editing screen allowing a user to edit the data table; and 
 display a delete option, the delete option allowing a user to delete the data table. 
 
     
     
       18. The system of  claim 13 , wherein the upload tool is further configured to display an expiration date, the expiration data indicating a date the data table will be deleted.