Label-based data representation I/O process and system

Abstract

A system and method for executing input/output (I/O) tasks for clients in a distributed computing system. One or more I/O requests made by a client are received. The operation instructions for the request data in the I/O requests are separated from the request data. A data representation called data label (or label) is created for executing operation instructions of the I/O requests. A data label corresponds to each of the I/O request and includes a unique identifier, information to the source and/or destination for the request data, and an operation instruction separated from the request data. The data label is pushed into a distributed label queue and is dispatched to an individual worker node according to a scheduling policy. The worker node executes the I/O tasks by executing the dispatched data label. The system and method can execute I/O tasks independently and decoupled from the client applications.

What is claimed is: 
     
       1. A method for executing input/output (I/O) requests from clients in a distributed computing system, the method comprising:
 receiving an input/output (I/O) request made by a client for request data in a distributed data system; 
 creating a data label corresponding to the I/O request, the data label including a unique identifier, a data pointer to a source and/or destination for the request data, and an operation instruction for the request data based upon the I/O request; 
 pushing the data label into a distributed label queue; 
 executing the operation instruction to the request data as a function of a scheduling policy of the distributed label queue. 
 
     
     
       2. The method of  claim 1 , further comprising dispatching the data label to a worker node for the executing the operation instruction according to the scheduling policy. 
     
     
       3. The method of  claim 2 , wherein the worker node is a storage server coordinated by a worker manager, wherein the worker manager is configured to monitor a worker node status. 
     
     
       4. The method of  claim 2 , further comprising:
 regulating the executing the operation instruction to the data by the worker node as a function of a system load and a worker node ability to execute the data label at a current point in time, wherein the worker node ability is evaluated by worker availability, worker remaining capacity, worker load, worker performance characteristics, worker energy signature, or combinations thereof. 
 
     
     
       5. The method of  claim 1 , further comprising temporarily placing content data corresponding to the request data in a data warehouse. 
     
     
       6. The method of  claim 5 , further comprising:
 creating a metadata entry comprising an inventory of the content data; and 
 storing the metadata entry in a catalog manager, wherein the catalog manager comprises a data inventory. 
 
     
     
       7. The method of  claim 1 , wherein the unique identifier includes a timestamp given at creation of the data label, and further comprising:
 the distributed label queue ordering operation of the data label as a function of the timestamp. 
 
     
     
       8. The method of  claim 1 , wherein the data label further comprises a status indicator. 
     
     
       9. The method of  claim 1 , further comprising forming the data label from a native or intercepted application programming interface (API) and from information of the I/O request. 
     
     
       10. The method of  claim 1 , further comprising:
 pushing the request data to a data-warehouse, wherein the data-warehouse comprises a distributed hashmap configured to temporarily hold the request data; and 
 creating a metadata entry for the request data pushed into the data-warehouse in a catalog embodied as an inventory maintained by a catalog manager, wherein the catalog manager comprising a second distributed hashmap and configured to maintain metadata information. 
 
     
     
       11. The method of  claim 1 , wherein creating the data label is implemented by a label manager using a configurable size parameter within a range of minimum and maximum values, wherein the values are dependent on a plurality of system parameters selected from the group consisting of memory page size, cache size, network type, type of destination storage, or combinations thereof, wherein the size parameter can be configured to be bypassed for predetermined I/O requests including synchronous reads. 
     
     
       12. The method of  claim 1 , further comprising executing the operation instruction independent and/or decoupled from the client. 
     
     
       13. The method of  claim 1 , further comprising:
 receiving the request data; 
 pushing content data of the request data to a data-warehouse configured to temporarily hold moving pieces of data; 
 creating a metadata entry for the content data in an inventory catalog, wherein the inventory catalog maintains both client and system metadata information; and 
 wherein the data label includes the unique identifier for the content data and the metadata entry. 
 
     
     
       14. A method for executing input/output (I/O) requests from clients in a distributed computing system, the method comprising:
 receiving request data for one of the I/O requests from a client; 
 separating the request data into content data and metadata; 
 tagging the content data with an operation instruction of the one of the I/O requests; 
 pushing the content data to a data-warehouse configured to temporarily hold data in the system; 
 creating a metadata entry in an inventory catalog for the content data in the data-warehouse, wherein the inventory catalog comprises a distributed hashmap and is configured to maintain both client and system metadata information; and 
 creating a data label comprising the operation instruction and a unique identifier for the content data and the metadata. 
 
     
     
       15. The method of  claim 14 , further comprising:
 pushing the data label into a distributed label queue; 
 dispatching the data label to a worker node according to a scheduling policy; and 
 executing the operation instruction of the data label by the worker node. 
 
     
     
       16. The method of  claim 15 , wherein the data label includes a timestamp given at creation, and further comprising:
 the distributed label queue ordering operation of the data label as a function of the timestamp and the scheduling policy. 
 
     
     
       17. The method of  claim 15 , wherein the worker node executes the operation instruction of the data label independently without direct interaction with the client. 
     
     
       18. A system for input/output (I/O) requests from clients in a distributed computing system, the system comprising:
 an application programming interface (API) configured to receive an I/O request from one of the clients; 
 a data label; 
 a distributed label queue; 
 a label manager configured to create the data label and push the data label to the distributed label queue, the data label including a unique identifier and a data pointer to a source and/or destination for a request data corresponding to the I/O request; 
 a worker node configured to execute the data label; and 
 a label dispatcher configured to dispatch the data label to the worker node according to a scheduling policy. 
 
     
     
       19. The system of  claim 18 , further comprising:
 a worker manager configured to manage the worker node according to a load of the system and an ability of a worker node to execute the data label at a current point in time. 
 
     
     
       20. The system of  claim 18 , further comprising:
 a data-warehouse configured to temporarily hold data, wherein the request data is uniquely identified and associated with the data label in the data-warehouse; and 
 an inventory catalog including client and system metadata information.