Efficient High Bandwidth Shared Memory Architectures for Parallel Machine Learning and AI Processing of Large Data Sets and Streams

Abstract

The present disclosure relates to systems and methods to implement efficient high-bandwidth shared memory systems particularly suited for parallelizing and operating large scale machine learning and AI computing systems necessary to efficiently process high volume data sets and streams.

1 . A method for implementing parallel processing with shared memory comprising:
 receiving, by at least one processor, a data stream object produced by one or more remote user devices;   saving the data stream object, by the at least one processor, as a sequence of individual elements into the shared memory;   analyzing, by the at least one processor, meta data describing processes to be applied to the data stream object;   determining, by the at least one processor, a plurality of independent parallel processing pipelines configured to perform the processes to be applied to the data stream object;
 wherein each independent parallel processing pipeline is configured to perform at least one particular process of the processes to be applied to the data stream object; 
   controlling, by the at least one processor, for a particular individual element of the sequence of individual elements, a plurality of independent parallel processing pipelines to concurrently and independently access the particular individual data element in the shared memory and perform the at least one particular process associated with each independent parallel processing pipeline; and   aggregating, by the at least one processor, in the shared memory, results for the particular individual element from the at least one particular process associated with each independent parallel processing pipeline.   
     
     
         2 . The method of  claim 1 , wherein the data stream object comprises at least one of a plurality of images, a video stream, and a sound stream. 
     
     
         3 . The method of  claim 1 , wherein the data stream object comprises meta data describing the processes to be applied to the data stream object. 
     
     
         4 . The method of  claim 1 , further comprising:
 analyzing, by the at least one processor, the data stream object for validity; and   applying, by the at least one processor, security measures to the data stream object.   
     
     
         5 . The method of  claim 1 , further comprising:
 controlling, by the at least one processor, for each individual element of the sequence of individual elements, the plurality of independent parallel processing pipelines to concurrently and independently access each individual data element in the shared memory;   controlling, by the at least one processor, for each individual element of the sequence of individual elements, the plurality of independent parallel processing pipelines to concurrently and independently perform the at least one particular process associated with each independent parallel processing pipeline; and   aggregating, by the at least one processor, in the shared memory, results for each individual element from the at least one particular process associated with each independent parallel processing pipeline.   
     
     
         6 . The method of  claim 5 , further comprising:
 generating, by the at least one processor, a results record for the particular individual element to record the results for the particular individual element from the at least one particular process associated with each independent parallel processing pipeline.   
     
     
         7 . The method of  claim 1 , further comprising:
 executing, by the at least one computing device, the process steps on the each of the individual elements using the sequence of algorithms.   
     
     
         8 . The method of  claim 1 , further comprising:
 generating, by the at least one computing device, a current state of the data stream object in the shared memory while the plurality of independent parallel processing pipelines concurrently and independently performs the at least one particular process associated with each independent parallel processing pipeline.   
     
     
         9 . The method of  claim 1 , further comprising:
 determining, by the at least one processor, the current state of the particular individual element upon performance of the at least one particular process.   
     
     
         10 . The method of  claim 1 , further comprising:
 balancing, by the at least one processor, a plurality of independent processing pipelines across the cluster of computing devices of using a predictive load balancer.   
     
     
         11 . A system for implementing parallel processing with shared memory comprising:
 at least one processor in communication with the shared memory;
 wherein the at least one processor is in communication with at least one non-transitory computer readable medium having software instructions stored thereon; 
 wherein the at least one processor is configured, upon execution of the software instructions, to:
 receive a data stream object produced by one or more remote user devices; 
 saving the data stream object, by the at least one processor, as a sequence of individual elements into the shared memory; 
 analyze meta data describing processes to be applied to the data stream object; 
 determine a plurality of independent parallel processing pipelines configured to perform the processes to be applied to the data stream object;
 wherein each independent parallel processing pipeline is configured to perform at least one particular process of the processes to be applied to the data stream object; 
 
 control for a particular individual element of the sequence of individual elements, a plurality of independent parallel processing pipelines to concurrently and independently access the particular individual data element in the shared memory and perform the at least one particular process associated with each independent parallel processing pipeline; and 
 aggregate in the shared memory, results for the particular individual element from the at least one particular process associated with each independent parallel processing pipeline. 
 
   
     
     
         12 . The system of  claim 11 , wherein the data stream object comprises at least one of a plurality of images, a video stream, and a sound stream. 
     
     
         13 . The system of  claim 11 , wherein the data stream object comprises meta data describing the processes to be applied to the data stream object. 
     
     
         14 . The system of  claim 11 , wherein the at least one processor is further configured, upon execution of the software instructions, to:
 analyze the data stream object for validity; and   apply security measures to the data stream object.   
     
     
         15 . The system of  claim 11 , wherein the at least one processor is further configured, upon execution of the software instructions, to:
 control for each individual element of the sequence of individual elements, the plurality of independent parallel processing pipelines to concurrently and independently access each individual data element in the shared memory;   control for each individual element of the sequence of individual elements, the plurality of independent parallel processing pipelines to concurrently and independently perform the at least one particular process associated with each independent parallel processing pipeline; and   aggregate in the shared memory, results for each individual element from the at least one particular process associated with each independent parallel processing pipeline.   
     
     
         16 . The system of  claim 15 , wherein the at least one processor is further configured, upon execution of the software instructions, to:
 generate a results record for the particular individual element to record the results for the particular individual element from the at least one particular process associated with each independent parallel processing pipeline.   
     
     
         17 . The system of  claim 11 , wherein the at least one processor is further configured, upon execution of the software instructions, to:
 execute the process steps on the each of the individual elements using the sequence of algorithms.   
     
     
         18 . The system of  claim 11 , wherein the at least one processor is further configured, upon execution of the software instructions, to:
 generate a current state of the data stream object in the shared memory while the plurality of independent parallel processing pipelines concurrently and independently performs the at least one particular process associated with each independent parallel processing pipeline.   
     
     
         19 . The system of  claim 11 , wherein the at least one processor is further configured, upon execution of the software instructions, to:
 determine the current state of the particular individual element upon performance of the at least one particular process.   
     
     
         20 . The system of  claim 11 , wherein the at least one processor is further configured, upon execution of the software instructions, to:
 balance a plurality of independent processing pipelines across the cluster of computing devices of using a predictive load balancer.