System, method, and computer-readable medium for iteratively dynamically assigning data objects to a plurality of processing modules of a database system

Abstract

A database system comprises a plurality of processing modules arranged to process data objects from a plurality of data object servers based upon a database query from a client computer system. A control task module is arranged to iteratively dynamically assign data objects from the plurality of data object servers to each of the plurality of processing modules based upon processing activity associated with the processing module during the database query from the client computer system. Alternatively, or in addition to, the control task module is arranged to iteratively dynamically assign data objects from the plurality of data object servers to each of the plurality of processing modules based upon a characteristic of the data objects to be dynamically assigned to the processing module during the database query from the client computer system.

What is claimed is: 
     
         1 . A database system comprising:
 a plurality of processing modules arranged to process data objects from a plurality of data object servers based upon a database query from a client computer system; and   a control task module arranged to iteratively dynamically assign data objects from the plurality of data object servers to each of the plurality of processing modules based upon processing activity associated with the processing module during the database query from the client computer system.   
     
     
         2 . A database system according to  claim 1 , wherein the control task module comprises a portion of one of the plurality of processing modules. 
     
     
         3 . A database system according to  claim 1  further comprising:
 a first communications network connected between the client computer system and each module; and 
 a second communications network connected between the plurality of data object servers and each module. 
 
     
     
         4 . A database system according to  claim 1 , wherein processing activity of each processing module during the database query comprises a number of output rows being generated by the processing module. 
     
     
         5 . A database system according to  claim 1 , wherein processing activity of each processing module during the database query comprises random processing activity associated with the processing module. 
     
     
         6 . A database system comprising:
 a plurality of processing modules arranged to process data objects from a plurality of data object servers based upon a database query from a client computer system; and   a control task module arranged to iteratively dynamically assign data objects from the plurality of data object servers to each of the plurality of processing modules based upon a characteristic of the data objects to be dynamically assigned to the processing module during the database query from the client computer system.   
     
     
         7 . A database system according to  claim 6 , wherein the control task module comprises a portion of one of the plurality of processing modules. 
     
     
         8 . A database system according to  claim 6  further comprising:
 a first communications network connected between the client computer system and each module; and 
 a second communications network connected between the plurality of data object servers and each module. 
 
     
     
         9 . A database system according to  claim 6 , wherein the characteristic of the data objects to be dynamically assigned to the processing module comprises size of one or more data objects to be assigned to the processing module. 
     
     
         10 . A database system according to  claim 6 , wherein the characteristic of the data objects to be dynamically assigned to the processing module comprises a collection of small batches of data objects to be dynamically assigned to the processing module. 
     
     
         11 . A method for assigning data objects to a plurality of processing modules of a database system to improve performance of the database system, the method comprising:
 receiving a database query from a client computer system; and   iteratively dynamically assigning data objects from a plurality of data object servers to each processing module based upon a select one of (i) processing activity of the processing module, and (ii) a characteristic of the data objects to be dynamically assigned to the processing module.   
     
     
         12 . A method according to  claim 11 , wherein receiving a database query from a client computer system includes:
 receiving a database query involving execution of a join operator.   
     
     
         13 . A method according to  claim 11 , wherein iteratively dynamically assigning data objects from a plurality of data object servers to each processing module based upon a select one of (i) processing activity of the processing module, and (ii) a characteristic of the data objects to be dynamically assigned to the processing module includes:
 iteratively dynamically assigning data objects from a plurality of data object servers to each processing module based upon both processing activity of the processing module and a characteristic of the data objects to be dynamically assigned to the processing module.   
     
     
         14 . A method according to  claim 11 , wherein iteratively dynamically assigning data objects from a plurality of data object servers to each processing module based upon a select one of (i) processing activity of the processing module, and (ii) a characteristic of the data objects to be dynamically assigned to the processing module includes:
 iteratively dynamically assigning data objects from a plurality of data object servers to each processing module based upon a number of output rows being generated by the processing module.   
     
     
         15 . A method according to  claim 11 , wherein iteratively dynamically assigning data objects from a plurality of data object servers to each processing module based upon a select one of (i) processing activity of the processing module, and (ii) a characteristic of the data objects to be dynamically assigned to the processing module includes:
 iteratively dynamically assigning data objects from a plurality of data object servers to each processing module based upon size of one or more data objects to be dynamically assigned to the processing module.   
     
     
         16 . A method according to  claim 11  further comprising:
 dynamically reassigning data objects from a processing module that is unable to process its assigned data objects to remaining processing modules so that the remaining processing modules can process the reassigned data objects. 
 
     
     
         17 . A method according to  claim 16 , wherein dynamically reassigning data objects from a processing module that is unable to process its assigned data objects to remaining processing modules so that the remaining processing modules can process the reassigned data objects includes:
 dynamically reassigning data objects to a processing module based upon processing speed of the processing module to which the data objects are reassigned.   
     
     
         18 . A method according to  claim 16 , wherein dynamically reassigning data objects from a processing module that is unable to process its assigned data objects to remaining processing modules so that the remaining processing modules can process the reassigned data objects includes:
 dynamically reassigning data objects to a processing module based upon a determination that the processing module from which data objects are being reassigned is classified as a straggler.   
     
     
         19 . A method according to  claim 11 , wherein the method is performed by a computer having a memory executing one or more program of instructions which are tangibly embodied in a storage medium readable by the computer. 
     
     
         20 . A method for a processing module iteratively processing data objects based upon a database query, the method comprising:
 requesting one or more data objects from a control task module;   receiving one or more data objects that have been iteratively dynamically assigned to the processing module by the control task module;   executing a step for the assigned one or more data objects;   providing processing activity information associated with the processing module to the control task module; and   repeating requesting of data objects from the control task module and providing processing activity information to the control task module until end of data stream associated with the database query.   
     
     
         21 . A method according to  claim 20 , wherein receiving one or more data objects that have been iteratively dynamically assigned to the processing module by the control task module includes:
 receiving only a part of a data object that is addressable without reading the whole object.