Prioritized execution of plans for obtaining and/or processing data

Abstract

In certain implementations, first and second plans (for obtaining data from one or more data sources and/or processing the obtained data) may be determined. The first and second plans may be executed. A first cost for fully executing the first plan may be estimated based on partial execution of the first plan, and a second cost for fully executing the second plan may be estimated based on partial execution of the second plan, where the first and second costs are estimated after the executions of the first and second plans have begun and before the executions of the first and second plans are completed. The execution of one of the first or second plans may be prioritized over at least the execution of the other one of the first or second plans based on a comparison of the estimated first and second costs.

What is claimed is: 
     
         1 . A computer-implemented method of prioritizing execution of plans for obtaining and/or processing data based on partial execution of the plans, the method being implemented by a computer system that includes one or more physical processors executing one or more computer program instructions that, when executed, perform the method, the method comprising:
 determining, by the computer system, at least a first plan and a second plan for obtaining data from one or more data sources and/or processing the obtained data;   executing, by the computer system, the first plan and the second plan;   estimating, by the computer system, based on partial execution of the first plan, a first cost for fully executing the first plan;   estimating, by the computer system, based on partial execution of the second plan, a second cost for fully executing the second plan, wherein the first cost and the second cost are estimated after the execution of the first plan and the execution of the second plan have begun and before the execution of the first plan and the execution of the second plan are completed; and   prioritizing, by the computer system, the execution of one of the first plan or the second plan over at least the execution of the other one of the first plan or the second plan based on a comparison of the estimated first cost and the estimated second cost.   
     
     
         2 . The method of  claim 1 , wherein the execution of the first plan is prioritized over at least execution of the second plan based on the comparison indicating that the estimated first cost is less than the estimated second cost, the method further comprising:
 continuing, by the computer system, the execution of the first plan based on the prioritization; and   suspending or canceling, by the computer system, the execution of the second plan based on the prioritization.   
     
     
         3 . The method of  claim 1 , further comprising:
 estimating, by the computer system, based on the partial execution of the first plan, a percentage of the first plan that has been executed; and   estimating, by the computer system, based on the partial execution of the second plan, a percentage of the second plan that has been executed,   wherein the first cost for fully executing the first plan is estimated based on a cost of the partial execution of the first plan and the percentage of the first plan that has been executed, and   wherein the second cost for fully executing the second plan is estimated based on a cost of the partial execution of the second plan and the percentage of the second plan that has been executed.   
     
     
         4 . The method of  claim 1 , wherein the first plan comprises obtaining results related to a data set from a data source, and the second plan comprises obtaining results related to the data set from the data source. 
     
     
         5 . The method of  claim 4 , wherein the first plan comprises utilizing a first path to obtain results related to the data set from the data source, and the second plan comprises utilizing a second path different than the first path to obtain results related to the data set from the data source. 
     
     
         6 . The method of  claim 1 , wherein the first plan comprises obtaining results related to a data set from a first data source before obtaining results related to the data set from a second data source, and the second plan comprises obtaining results related to the data set from the second data source before obtaining results related to the data set from the first data source. 
     
     
         7 . The method of  claim 1 , wherein the first plan comprises obtaining results related to a data set from a data source that is not utilized in the second plan for obtaining results related to the data set. 
     
     
         8 . The method of  claim 1 , further comprising:
 estimating, based on the partial execution of the first plan, a first delay associated with presenting a set of results related to a data set when the first plan is used; and   estimating, based on the partial execution of the second plan, a second delay associated with presenting a set of results related to the data set when the second plan is used,   wherein the first cost for fully executing the first plan is estimated based on the estimated first delay, and   wherein the second cost for fully executing the second plan is estimated based on the estimated second delay.   
     
     
         9 . The method of  claim 1 , further comprising:
 estimating, based on the partial execution of the first plan, first delays between updates to a presentation of a data set to a user when the first plan is used; and   estimating, based on the partial execution of the second plan, second delays between updates to a presentation of a data set to a user when the second plan is used,   wherein the first cost for fully executing the first plan is estimated based on the estimated first delays, and   wherein the second cost for fully executing the second plan is estimated based on the estimated second delays.   
     
     
         10 . The method of  claim 1 , wherein estimating the first cost is based on predefined performance metrics associated with a first data source that is utilized in the first plan, and wherein estimating the second cost is based on predefined performance metrics associated with a second data source that is utilized in the second plan. 
     
     
         11 . The method of  claim 10 , wherein the predefined performance metrics comprise one or both of latency and throughput of data through the data sources. 
     
     
         12 . The method of  claim 1 , wherein estimating the first cost is based on historical cost information associated with one or more aspects of the first plan, and wherein estimating the second cost is based on historical cost information associated with one or more aspects of the second plan. 
     
     
         13 . The method of  claim 1 , wherein the first plan and the second plan are executed in parallel before the prioritization. 
     
     
         14 . The method of  claim 13 , wherein the first cost is estimated based on the partial execution of the first plan that occurred in parallel with the execution of the second plan, and the second cost is estimated based on the partial execution of the second plan that occurred in parallel with the execution of the first plan. 
     
     
         15 . A system for prioritizing execution of plans for obtaining and/or processing data based on partial execution of the plans, the system comprising:
 one or more physical processors programmed to execute one or more computer program instructions that, when executed, cause the one or more physical processors to:
 determine at least a first plan and a second plan for obtaining data from one or more data sources and/or processing the obtained data; 
 execute the first plan and the second plan; 
 estimate, based on partial execution of the first plan, a first cost for fully executing the first plan; 
 estimate, based on partial execution of the second plan, a second cost for fully executing the second plan, wherein the first cost and the second cost are estimated after the execution of the first plan and the execution of the second plan have begun and before the execution of the first plan and the execution of the second plan are completed; and 
 prioritize the execution of one of the first plan or the second plan over at least the execution of the other one of the first plan or the second plan based on a comparison of the estimated first cost and the estimated second cost. 
   
     
     
         16 . The system of  claim 15 , wherein the execution of the first plan is prioritized over at least execution of the second plan based on the comparison indicating that the estimated first cost is less than the estimated second cost, and wherein the one or more physical processors are further caused to:
 continue the execution of the first plan based on the prioritization; and   suspend or cancel the execution of the second plan based on the prioritization.   
     
     
         17 . The system of  claim 15 , wherein the one or more physical processors are further caused to:
 estimate, based on the partial execution of the first plan, a percentage of the first plan that has been executed; and   estimate, based on the partial execution of the second plan, a percentage of the second plan that has been executed,   wherein the first cost for fully executing the first plan is estimated based on a cost of the partial execution of the first plan and the percentage of the first plan that has been executed, and   wherein the second cost for fully executing the second plan is estimated based on a cost of the partial execution of the second plan and the percentage of the second plan that has been executed.   
     
     
         18 . The system of  claim 15 , wherein first plan comprises obtaining results related to a data set from a data source and comprises utilizing a first path to obtain results related to the data set from the data source, and the second plan comprises obtaining results related to the data set from the data source and utilizing a second path different than the first path to obtain results related to the data set from the data source. 
     
     
         19 . The system of  claim 15 , wherein the first plan comprises obtaining results related to a data set from a first data source before obtaining results related to the data set from a second data source, and the second plan comprises obtaining results related to the data set from the second data source before obtaining results related to the data set from the first data source. 
     
     
         20 . The system of  claim 15 , wherein the first plan comprises obtaining results related to a data set from a data source that is not utilized in the second plan for obtaining results related to the data set.