Run time estimation system optimization

Abstract

Methods, systems, and computer program products for training an optimized time estimation system for completing a data processing job to be run on a data processing device that operates within a distributed processing system having a range of platforms. Embodiments include creating a prediction algorithm based upon retrieved operational parameters associated with a data processing job. Embodiments also include retrieving further operational parameters associated with the data processing job. Embodiments include updating the prediction algorithm based on the further operational parameters, in which the prediction algorithm is updated by modifying parameter values associated with variable parameters of the prediction algorithm.

1 - 62 . (canceled) 
     
     
         63 . A method, implemented at a computer system that includes one or more processors, of training an optimized time estimation system for completing a data processing job to be run on at least one data processing device that operates within a distributed processing system having a range of platforms, the method including:
 creating a prediction algorithm based upon retrieved operational parameters associated with a data processing job;   retrieving further operational parameters associated with the data processing job; and   updating the prediction algorithm based on the further operational parameters, wherein the prediction algorithm is updated by modifying parameter values associated with variable parameters of the prediction algorithm.   
     
     
         64 . The method of  claim 63 , the method further including:
 determining a sum of errors based on a cost of using current weight values; and   determining a variation in cost based on variations in the parameter values.   
     
     
         65 . The method of  claim 64 , wherein the cost of using current parameter values is determined by calculating a sum of all errors made when processing previous data processing jobs. 
     
     
         66 . The method of  claim 64 , the method further including:
 determining a gradient based on a rate of change of the cost and using local approximation to determine the change in cost based on a change in weight values.   
     
     
         67 - 75 . (canceled) 
     
     
         76 . The method of  claim 63 , wherein the further operational parameters are based on a prior execution of the data processing job. 
     
     
         77 . The method of  claim 63 , wherein the further operational parameters are based on an accuracy of a prior execution of the prediction algorithm. 
     
     
         78 . The method of  claim 63 , wherein further operational parameters are based on use of a training data set with the data processing job. 
     
     
         79 . A computer system, comprising:
 one or more hardware processors; and   one or more computer-readable media having stored thereon computer-executable instructions that are executable by the one or more hardware processors and that are structured to configure the computer system to train an optimized time estimation system for completing a data processing job to be run on at least one data processing device that operates within a distributed processing system having a range of platforms, the computer-executable instructions including computer-executable instructions that are structured to configure the computer system to:
 create a prediction algorithm based upon retrieved operational parameters associated with a data processing job; 
 retrieve further operational parameters associated with the data processing job; and 
 update the prediction algorithm based on the further operational parameters, wherein the prediction algorithm is updated by modifying parameter values associated with variable parameters of the prediction algorithm. 
   
     
     
         80 . The computer system of  claim 79 , wherein the computer-executable instructions also include computer-executable instructions that are structured to configure the computer system to:
 determine a sum of errors based on a cost of using current weight values; and   determine a variation in cost based on variations in the parameter values.   
     
     
         81 . The computer system of  claim 80 , wherein the computer-executable instructions also include computer-executable instructions that are structured to configure the computer system to determine the cost of using current parameter values by calculating a sum of all errors made when processing previous data processing jobs. 
     
     
         82 . The computer system of  claim 80 , wherein the computer-executable instructions also include computer-executable instructions that are structured to configure the computer system to:
 determine a gradient based on a rate of change of the cost and using local approximation to determine the change in cost based on a change in weight values.   
     
     
         83 . The computer system of  claim 79 , wherein the further operational parameters are based on a prior execution of the data processing job. 
     
     
         84 . The computer system of  claim 79 , wherein the further operational parameters are based on an accuracy of a prior execution of the prediction algorithm. 
     
     
         85 . The computer system of  claim 79 , wherein further operational parameters are based on use of a training data set with the data processing job. 
     
     
         86 . A computer program product comprising one or more hardware storage devices having stored thereon computer-executable instructions that are executable by one or more processors of a computer system and that are structured to configure the computer system to train an optimized time estimation system for completing a data processing job to be run on at least one data processing device that operates within a distributed processing system having a range of platforms, including computer-executable instructions that are structured to configure the computer system to:
 create a prediction algorithm based upon retrieved operational parameters associated with a data processing job;   retrieve further operational parameters associated with the data processing job; and   update the prediction algorithm based on the further operational parameters, wherein the prediction algorithm is updated by modifying parameter values associated with variable parameters of the prediction algorithm.   
     
     
         87 . The computer program product of  claim 86 , wherein the computer-executable instructions also include computer-executable instructions that are structured to configure the computer system to:
 determine a sum of errors based on a cost of using current weight values; and   determine a variation in cost based on variations in the parameter values.   
     
     
         88 . The computer program product of  claim 87 , wherein the computer-executable instructions also include computer-executable instructions that are structured to configure the computer system to determine the cost of using current parameter values by calculating a sum of all errors made when processing previous data processing jobs. 
     
     
         89 . The computer program product of  claim 87 , wherein the computer-executable instructions also include computer-executable instructions that are structured to configure the computer system to:
 determine a gradient based on a rate of change of the cost and using local approximation to determine the change in cost based on a change in weight values.   
     
     
         90 . The computer program product of  claim 86 , wherein the further operational parameters are based on a prior execution of the data processing job. 
     
     
         91 . The computer program product of  claim 86 , wherein the further operational parameters are based on an accuracy of a prior execution of the prediction algorithm.