System for and method of multiple machine learning model aggregation

Abstract

Systems, methods, and computer-readable storage media for aggregating the outputs of multiple machine learning models, then using the output of yet another machine learning model as a multiplier to obtain a final prediction. A system can receiving a plurality of data sets, each data set being associated with at least one data type, and train machine learning models, each model associated with one or more of the different data types. Upon execution, the multiple machine learning models can each produce a prediction which is aggregated together to form an aggregated prediction. The multiplier from the additional machine learning model can then be applied to the aggregated prediction, resulting in a final prediction.

We claim: 
     
         1 . A method comprising:
 receiving, at a computer system, a plurality of data sets, wherein each data set within the plurality of data sets is associated with at least one data type within a plurality of data types;   training, via at least one processor of the computer system using the plurality of data sets, a plurality of machine learning models,
 wherein each machine learning model in the plurality of machine learning models is configured to, upon execution, generate a transportation prediction; and 
 wherein each machine learning model in the plurality of machine learning models is trained using a data set within the plurality of data sets; 
   receiving, at the computer system, real-time data type values associated with the plurality of data types;   executing, via the at least one processor, the plurality of machine learning models using the real-time data type values as inputs, resulting in a plurality of predictions; and   generating, via the at least one processor aggregating the plurality of predictions, a final prediction.   
     
     
         2 . The method of  claim 1 , further comprising:
 receiving, at the computer system, past industry-specific indicators;   training, via the at least one processor using the industry-specific indicators, a machine learning multiplier model;   receiving, at the computer system, current industry-specific indicators; and   executing, via the at least one processor, the machine learning multiplier model using the current industry-specific indicators as input, resulting in a multiplier,   wherein the final prediction is further generated using the multiplier.   
     
     
         3 . The method of  claim 2 , further comprising:
 receiving, at the computer system after generating the final prediction, an actual result;   determining, via the at least one processor, that the actual result is distinct from the final prediction, resulting in a difference;   retraining, via the at least one processor, the machine learning multiplier model using the industry-specific indicators and the difference, resulting in an updated machine learning multiplier model; and   retraining, via the at least one processor, the plurality of machine learning models using the plurality of data sets and the difference, resulting in an updated plurality of machine learning models,   wherein the updated machine learning multiplier model and the updated plurality of machine learning models are used in future iterations.   
     
     
         4 . The method of  claim 3 , wherein the retraining of the machine learning multiplier model occurs more frequently than the retraining of the plurality of machine learning models. 
     
     
         5 . The method of  claim 3 , wherein the retraining of the machine learning multiplier model and the retraining of the plurality of machine learning models occur upon a predetermined number of iterations of the generating of the final prediction occurring. 
     
     
         6 . The method of  claim 1 , wherein the final prediction is a freight transportation price. 
     
     
         7 . The method of  claim 1 , wherein the plurality of data types comprise: holiday information, seasonal data, and fuel price data. 
     
     
         8 . A system comprising:
 at least one processor; and   a non-transitory computer-readable storage medium having instructions which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:   receiving a plurality of data sets, wherein each data set within the plurality of data sets is associated with a different data type within a plurality of data types;   training, using the plurality of data sets, a plurality of machine learning models,
 wherein each machine learning model in the plurality of machine learning models is configured to, upon execution, generate a transportation prediction; and 
 wherein each machine learning model in the plurality of machine learning models is trained using a data set within the plurality of data sets; 
   receiving real-time data type values associated with the plurality of data types;   executing the plurality of machine learning models using the real-time data type values as inputs, resulting in a plurality of predictions; and   generating, by aggregating the plurality of predictions, a final prediction.   
     
     
         9 . The system of  claim 8 , the non-transitory computer-readable storage medium having additional instructions stored which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
 receiving past industry-specific indicators;   training, using the industry-specific indicators, a machine learning multiplier model;   receiving current industry-specific indicators; and   executing the machine learning multiplier model using the current industry-specific indicators as input, resulting in a multiplier,   wherein the final prediction is further generated using the multiplier.   
     
     
         10 . The system of  claim 9 , the non-transitory computer-readable storage medium having additional instructions stored which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
 receiving, after generating the final prediction, an actual result;   determining that the actual result is distinct from the final prediction, resulting in a difference;   retraining the machine learning multiplier model using the industry-specific indicators and the difference, resulting in an updated machine learning multiplier model; and   retraining the plurality of machine learning models using the plurality of data sets and the difference, resulting in an updated plurality of machine learning models,   wherein the updated machine learning multiplier model and the updated plurality of machine learning models are used in future iterations.   
     
     
         11 . The system of  claim 10 , wherein the retraining of the machine learning multiplier model occurs more frequently than the retraining of the plurality of machine learning models. 
     
     
         12 . The system of  claim 10 , wherein the retraining of the machine learning multiplier model and the retraining of the plurality of machine learning models occur upon a predetermined number of iterations of the generating of the final prediction occurring. 
     
     
         13 . The system of  claim 8 , wherein the final prediction is a freight transportation price. 
     
     
         14 . The system of  claim 8 , wherein the plurality of data types comprise: holiday information, seasonal data, and fuel price data. 
     
     
         15 . A non-transitory computer-readable storage medium having instructions which, when executed by at least one processor, cause the at least one processor to perform operations comprising:
 receiving a plurality of data sets, wherein each data set within the plurality of data sets is associated with a different data type within a plurality of data types;   training, using the plurality of data sets, a plurality of machine learning models,
 wherein each machine learning model in the plurality of machine learning models is configured to, upon execution, generate a transportation prediction; and 
 wherein each machine learning model in the plurality of machine learning models is trained using a data set within the plurality of data sets; 
   receiving real-time data type values associated with the plurality of data types;   executing the plurality of machine learning models using the real-time data type values as inputs, resulting in a plurality of predictions; and   generating, by aggregating the plurality of predictions, a final prediction.   
     
     
         16 . The non-transitory computer-readable storage medium of  claim 8 , having additional instructions stored which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
 receiving past industry-specific indicators;   training, using the industry-specific indicators, a machine learning multiplier model;   receiving current industry-specific indicators; and   executing the machine learning multiplier model using the current industry-specific indicators as input, resulting in a multiplier,   wherein the final prediction is further generated using the multiplier.   
     
     
         17 . The non-transitory computer-readable storage medium of  claim 16 , having additional instructions stored which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
 receiving, after generating the final prediction, an actual result;   determining that the actual result is distinct from the final prediction, resulting in a difference;   retraining the machine learning multiplier model using the industry-specific indicators and the difference, resulting in an updated machine learning multiplier model; and   retraining the plurality of machine learning models using the plurality of data sets and the difference, resulting in an updated plurality of machine learning models,   wherein the updated machine learning multiplier model and the updated plurality of machine learning models are used in future iterations.   
     
     
         18 . The non-transitory computer-readable storage medium of  claim 17 , wherein the retraining of the machine learning multiplier model occurs more frequently than the retraining of the plurality of machine learning models. 
     
     
         19 . The non-transitory computer-readable storage medium of  claim 17 , wherein the retraining of the machine learning multiplier model and the retraining of the plurality of machine learning models occur upon a predetermined number of iterations of the generating of the final prediction occurring. 
     
     
         20 . The non-transitory computer-readable storage medium of  claim 15 , wherein the final prediction is a freight transportation price.