Distributed computing

Abstract

A distributed computing system manages execution of jobs and their associated tasks. Multiple scheduling strategies respect job priority preferences. A graphical user interface allows viewing of job status information and on-the-fly modification of job priorities.

1 . A distributed computing system, comprising: 
 a plurality of engines that perform assigned computational tasks;    at least one broker that manages jobs submitted for execution by client applications, each job including (i) one or more tasks, (ii) ajob id, and (iii) ajob priority, the task(s) associated with each job organized into at least three groups including (a) waiting tasks that have not yet been assigned to an engine for execution, (b) pending tasks that have been assigned to an engine for execution, but have not yet completed executing, and (c) completed tasks that have completed execution on an assigned engine, each task having at least (i) a task id and (ii) a task priority;    the at least one broker configured to assign waiting task(s) to available engine(s); and,    a graphical user interface capable of (i) displaying status information for jobs and (ii) allowing a user to modify job priorities.    
   
   
       2 . A distributed computing system, comprising: 
 a plurality of engines that perform assigned computational tasks;    at least one primary broker that manages jobs submitted for execution by client applications, each job including (i) one or more tasks, (ii) a job id, and (iii) a job priority, the task(s) associated with each job organized into at least three groups including (a) waiting tasks that have not yet been assigned to an engine for execution, (b) pending tasks that have been assigned to an engine for execution, but have not yet completed executing, and (c) completed tasks that have completed execution on an assigned engine, each task having at least (i) a task id and (ii) a task priority;    the at least one primary broker configured to assign waiting task(s) to available engine(s); and,    at least one failover broker that performs the fimctions of the primary broker in the event of a primary broker failure.    
   
   
       3 . A distributed computing system, as defined in  claim 2 , wherein the at least one failover broker operates on a different server than the primary broker.  
   
   
       4 . A distributed computing system, as defined in  claim 3 , further comprising: 
 a graphical user interface capable of (i) displaying status information for jobs and (ii) allowing a user to modify job priorities.    
   
   
       5 . A distributed computing system, as defined in  claim 3 , wherein said at least one broker includes a scheduler that operates in at least two modes.  
   
   
       6 . A distributed computing system, as defined in  claim 5 , wherein said at least two modes include (i) a first mode that gives greater execution preference to higher priority jobs and (ii) a second mode that gives less execution preference to higher priority jobs.  
   
   
       7 . A distributed computing system, as defined in  claim 4 , further comprising at least one cache that provides a data storage area for data used by tasks.  
   
   
       8 . A distributed computing system, as defined in  claim 7 , wherein the cache permits data sharing among jobs.  
   
   
       9 . A distributed computing system, as defined in  claim 5 , further comprising at least one cache that provides a data storage area for data used by tasks.  
   
   
       10 . A distributed computing system, as defined in  claim 5 , further comprising a batch administration tool that manages execution of batch job(s).  
   
   
       11 . A distributed computing system, comprising: 
 at least one client application that submits job(s) for execution, each job including (i) one or more tasks and (ii) a job priority;    a plurality of engines that execute assigned tasks;    at least one broker that manages jobs submitted for execution by client application(s), the task(s) associated with each job organized into at least three groups including (a) waiting tasks that have not yet been assigned to an engine for execution, (b) pending tasks that have been assigned to an engine for execution, but have not yet completed executing, and (c) completed tasks that have completed execution on an assigned engine, each task having an associated task priority, the at least one broker configured to assign waiting task(s) to available engine(s); and,    a messaging API that sends input data to, and receives output data from, executing task(s).    
   
   
       12 . A distributed computing system, as defined in  claim 11 , wherein each job further includes an associated job id.  
   
   
       13 . A distributed computing system, as defined in  claim 12 , wherein each task further includes an associated task id.  
   
   
       14 . A distributed computing system, as defined in  claim 12 , further comprising a graphical user interface capable of (i) displaying status information for jobs and (ii) allowing a user to modify job priorities.  
   
   
       15 . A distributed computing system, as defined in  claim 11 , further comprising at least one cache that provides a data storage area for data used by tasks.  
   
   
       16 . A distributed computing system, as defined in  claim 15 , wherein the cache permits data sharing among jobs.  
   
   
       17 . A distributed computing system, as defined in  claim 11 , wherein said at least one broker includes a job scheduler that implements at least two scheduling strategies.  
   
   
       18 . A distributed computing system, as defined in  claim 17 , wherein a first scheduling strategy allocates a larger share of available resource(s) to higher priority job(s) than a second scheduling strategy.  
   
   
       19 . A distributed computing system, as defined in  claim 11 , further comprising at least one failover broker that performs the functions of the broker if the broker fails to perform.  
   
   
       20 . A distributed computing system, as defined in  claim 19 , wherein the broker and the at least one failover broker operate on different servers.  
   
   
       21 . A method for operating a distributed computing system, the method comprising: 
 using a plurality of engines to execute assigned computational tasks;    using at least one broker to manage jobs submitted for execution by client applications, each job including (i) one or more tasks, (ii) a job id, and (iii) a job priority;    organizing the task(s) associated with each job into at least three categories, including (a) waiting tasks that have not yet been assigned to an engine for execution, (b) pending tasks that have been assigned to an engine for execution, but have not yet completed executing, and (c) completed tasks that have completed execution on an assigned engine;    configuring the at least one broker to assign waiting task(s) to available engine(s); and,    providing a graphical user interface that (i) displays status information for jobs and (ii) allows a user to modify job priorities.    
   
   
       22 . A method for operating a distributed computing system, as defined in  claim 21 , further comprising: 
 associating a task id and a task priority with each task.    
   
   
       23 . A method for operating a distributed computing system, the method comprising: 
 executing assigned computational tasks using a plurality of network-connected engines;    using at least one network-connected primary broker to manage jobs submitted for execution by client applications, each job including (i) one or more tasks, (ii) ajob id, and (iii) ajob priority; and,    if the at least one primary broker encounters a failure, automatically replacing the primary broker with at least one failover broker that performs the ftmctions of the primary broker.    
   
   
       24 . A method for operating a distributed computing system, as defined in  claim 23 , further comprising: 
 organizing the task(s) associated with each job into at least three groups, including (a) waiting tasks that have not yet been assigned to an engine for execution, (b) pending tasks that have been assigned to an engine for execution, but have not yet completed executing, and (c) completed tasks that have completed execution on an assigned engine; and,    using the at least one primary broker or the at least one failover broker to assign waiting task(s) to available engine(s).    
   
   
       25 . A method for operating a distributed computing system, as defined in  claim 23 , further comprising associating a task id and (ii) a task priority with each task.  
   
   
       26 . A method for operating a distributed computing system, as defined in  claim 23 , wherein the at least one failover broker operates on a different server than the primary broker.  
   
   
       27 . A method for operating a distributed computing system, as defined in  claim 26 , further comprising: 
 providing a graphical user interface capable of (i) displaying status information for jobs and (ii) allowing a user to modify job priorities.    
   
   
       28 . A method for operating a distributed computing system, as defined in  claim 23 , further comprising providing at least two scheduling strategies.  
   
   
       29 . A method for operating a distributed computing system, as defined in  claim 28 , wherein a first scheduling strategy gives greater execution preference to higher priority jobs and a second scheduling strategy gives less execution preference to higher priority jobs.  
   
   
       30 . A method for operating a distributed computing system, as defined in  claim 23 , further comprising caching data used by tasks.  
   
   
       31 . A method for operating a distributed computing system, as defined in  claim 30 , further comprising sharing cached data among jobs.  
   
   
       32 . A method for operating a distributed computing system, as defined in  claim 23 , further comprising managing execution of batch job(s) using a batch administration tool.  
   
   
       33 . A method for operating a distributed computing system, comprising: 
 allowing a client application to submits job(s) for execution, each job including (i) one or more tasks and (ii) a job priority;    using a plurality of engines to execute assigned tasks;    using at least one broker to manage execution of submitted job(s) by (1) maintaining the task(s) associated with each job in at least three groups, including (a) waiting tasks that have not yet been assigned to an engine for execution, (b) pending tasks that have been assigned to an engine for execution, but have not yet completed executing, and (c) completed tasks that have completed execution on an assigned engine, and (2) using associated task priorities to assign waiting task(s) to available engine(s); and, using a messaging API to send input data to, and receive output data from, executing task(s).    
   
   
       34 . A method for operating a distributed computing system, as defined in  claim 33 , further comprising associating a job id with each job.  
   
   
       35 . A method for operating a distributed computing system, as defined in  claim 34 , further comprising associating a task id with each task.  
   
   
       36 . A method for operating a distributed computing system, as defined in  claim 34 , further comprising providing a graphical user interface capable of (i) displaying status information for jobs and (ii) allowing a user to modify job priorities.  
   
   
       37 . A method for operating a distributed computing system, as defined in  claim 33 , further comprising using at least one cache as a data storage area for data used by tasks.  
   
   
       38 . A method for operating a distributed computing system, as defined in  claim 37 , further comprising using the cache to share data among jobs.  
   
   
       39 . A method for operating a distributed computing system, as defined in  claim 34 , further comprising using at least two scheduling strategies to assign available resource(s) to waiting task(s).  
   
   
       40 . A method for operating a distributed computing system, as defined in  claim 39 , wherein a first scheduling strategy allocates a larger share of available resource(s) to task(s) associated with higher priority job(s) than a second scheduling strategy.  
   
   
       41 . A method for operating a distributed computing system, as defined in  claim 40 , further comprising using at least one failover broker to perform the functions of the broker if the broker fails to perform.  
   
   
       42 . A method for operating a distributed computing system, as defined in  claim 41 , wherein the broker and the at least one failover broker operate on different servers.