Systems and methods to execute efficiently a plurality of machine learning processes

Abstract

Described herein are systems and methods for executing efficiently, in real-time, a plurality of machine learning processes. In one embodiment, a computing platform with multiple compute elements receives multiple data streams, each such stream associated with its own respective machine learning process. Each machine learning process is operative to use its data stream as input to train, in real-time, a respective mathematical model. Each of the processes has peaks and dips in processing demands. The system re-allocates, in real-time, compute elements from the processes with lower processing demands to processes with higher processing demands, thereby handling all of the multiple processes on-the-fly, preventing peak demands from causing the system to stall, and reducing overall the computational resources required by the system.

1 . A method for efficiently executing a plurality of machine learning processes, comprising:
 receiving, in a computing platform comprising a plurality of compute elements, a plurality of streams of data sets;   continuously training a plurality of mathematical models using respectively the plurality of streams acting as inputs, in which said continuous training is executed respectively as a plurality of machine learning processes in conjunction with the compute elements;   detecting, in the computing platform, during executing of the plurality of machine learning processes, a temporary condition in which one of the continuous training of one of the mathematical models is lagging behind the respective stream as a result of a temporary computational state associated with the mathematical model and the respective stream; and   at least temporarily re-allocating some of the plurality of compute elements away from at least some of the continuous training currently requiring less computational resources, and toward boosting performance of said continuous training which lags behind the stream, thereby allowing the computing platform to cope with the temporary condition.   
     
     
         2 . The method of  claim 1 , wherein the temporary computational state is a state in which the respective mathematical model has evolved into a certain state of the mathematical model which inherently requires more computational resources to process the respective data sets of the respective stream. 
     
     
         3 . The method of  claim 2 , wherein said boosting of performance comprises:
 changing the respective mathematical model from the certain state of the mathematical model into a previous state of the mathematical model; and   re-training the respective mathematical model, using the respective data sets, thereby converging to a new-state of the mathematical model, which is different than said certain state, thereby eliminating the temporary computational state.   
     
     
         4 . The method of  claim 3 , wherein said re-training comprises:
 using the respective data sets, repeatedly, a plurality of times, each time producing a different alternative state of the mathematical model, thereby producing a plurality of alternative forms of the mathematical model; and   selecting the best one of the plurality of alternative forms as the new state of the mathematical model.   
     
     
         5 . The method of  claim 2 , wherein said boosting of performance comprises:
 distributing the continuous training of the certain state of the mathematical model among those of the compute elements re-allocated toward said boosting, thereby converging into a next state of the mathematical model which eliminates the temporary computational state.   
     
     
         6 . The method of  claim 1 , wherein the temporary computational state is a state in which the respective mathematical model is no longer valid in view of a certain change in the data sets of the respective stream. 
     
     
         7 . The method of  claim 6 , wherein said boosting of performance comprises:
 resetting the respective mathematical model into an initial mathematical model as a result of said change; and   re-training the initial mathematical model, using the respective data sets, thereby converging to a new state of the mathematical model, thereby eliminating the temporary computational state.   
     
     
         8 . The method of  claim 1 , wherein the respective mathematical model is a prediction model, a classification model, or a clustering model, in which the respective continuous training of the model, in view of the respective stream, is done using a technique associated with a gradient-descent or stochastic-gradient-descent, in which said temporary computational state is a state associated with poor convergence of the gradient-descent or stochastic-gradient-descent technique. 
     
     
         9 . The method of  claim 1 , wherein the respective mathematical model is a neural network model, in which the respective continuous training of the model, in view of the respective stream, is done using a technique associated with deep learning, in which said temporary computational state is a state associated with a need to either increase a complexity of the neural network model or increase a number of layers associated with the neural network model. 
     
     
         10 . A system operative to execute in real-time a plurality of machine learning processes, comprising:
 a computing platform comprising a plurality of compute elements; and   a plurality of streams of data sets, received in the computing platform, and associated with, respectively, a plurality of machine learning processes, in which each of the machine learning processes is operative to train, in real-time, a respective mathematical model using the respective stream as a real-time input, and in which each of the machine learning processes is characterized by having peak and dip demands for computational resources during progression of the training of the respective mathematical model;   wherein: the system is configured to re-allocate, in real-time, the compute elements to the different machine learning processes according to said demands, such that all of the peak demands are handled by the system on-the-fly by re-allocating more compute elements to the machine learning processes having the peak demands, and thereby preventing the peak demands from stalling the system or otherwise causing the system to fail in said training in real-time of the mathematical models.   
     
     
         11 . The system of  claim 10 , wherein the plurality of machine learning processes are uncorrelated, therefore causing the respective peak demands to be uncorrelated in time, thereby enabling the system to avoid said stalling. 
     
     
         12 . The system of  claim 11 , wherein as a result of the plurality of machine learning processes being uncorrelated, the system is able to avoid said stalling even when a total processing power of the computing platform is less that a certain processing power needed to handle all peak demands simultaneously. 
     
     
         13 . The system of  claim 12 , wherein as a result of an inherent nature of machine learning processes in general, the peak demand for computational resources is significantly higher than both the dip demand and an average demand for computational resources, and consequently, statistically speaking, as a result of said re-allocation in real time, the system is able prevent said stalling even when the total processing power of the computing platform is just at, or slightly higher than, the level of a processing power needed to merely handle all average demands simultaneously. 
     
     
         14 . The system of  claim 13 , wherein as a result of said inherent nature:
 the ratio between the peak demand and the dip demand for computational resources is above one hundred to one (above 100:1);   the ratio between the average demand and the dip demand for computational resources is below two to one (below 2:1);   and therefore the system is able to prevent said stalling with less than two percent (2%) of the computational resources that would have otherwise been needed in a case that said re-allocation in real-time was not available.   
     
     
         15 . A method for significantly reducing processing resources needed to execute a plurality of real-time machine learning processes, comprising:
 performing, in a computing platform comprising a plurality of compute elements, a plurality of real-time machine learning processes associated with, respectively, a plurality of real-time streams, in which each of the machine learning processes is inherently characterized by having a peak demand for computational resources that is significantly higher than an average demand for computational resources;   re-allocating, by the computing platform, in real-time, the compute elements to the different machine learning processes according to said demands, such that all of the peak demands are handled by the computing platform on-the-fly by re-allocating more compute elements to the machine learning processes currently having the peak demands, and thereby preventing the peak demands from stalling the system; and   consequently, significantly increasing utilization of the plurality of compute elements, thereby decreasing the actual number of said compute elements needed to achieve said prevention of stalling the system.   
     
     
         16 . The method of  claim 15 , wherein the plurality of real-time machine learning processes are uncorrelated, therefore causing the respective peak demands to be uncorrelated in time, thereby enabling the system to achieve said decrease in actual number of said compute elements needed to achieve said prevention of stalling the system. 
     
     
         17 . The method of  claim 16 , wherein the plurality of machine learning processes are uncorrelated as a result of the streams being originated by different uncorrelated sources. 
     
     
         18 . The method of  claim 16 , wherein the plurality of machine learning processes are uncorrelated as a result of the streams being made intentionally uncorrelated by adapting, rearranging, reordering, or otherwise manipulating a single real-time stream into said plurality of streams which are uncorrelated. 
     
     
         19 . The method of  claim 16 , wherein the plurality of machine learning processes are uncorrelated as a result of breaking a single master machine learning process into said plurality of machine learning processes. 
     
     
         20 . A computer-implemented method, comprising:
 executing, by a first processing device, a code sequence,
 wherein the code sequence includes a first set of commands, a second set of commands, and a third command, and 
 wherein executing the code sequence includes executing the first set of commands and deferring execution of the second set of commands until the third command is encountered; 
   encountering, by the first processing device, the third command;   based on encountering the third command,
 compiling, by the first processing device, the second set of commands from a first representation in a first high-level programming language to a second representation in a second high-level programming language, and 
 compiling, by the first processing device, the second representation of the second set of commands in the second high-level programming language to an executable representation; and 
   sending, by the first processing device, the executable representation of the second set of commands to a remote processing device for execution.