Minimizing memory reads and increasing performance of a neural network environment using a directed line buffer

Abstract

The performance of a neural network (NN) and/or deep neural network (DNN) can limited by the number of operations being performed as well as management of data among the various memory components of the NN/DNN. Using a directed line buffer that operatively inserts one or more shifting bits in data blocks to be processed, data read/writes to the line buffer can be optimized for processing by the NN/DNN thereby enhancing the overall performance of a NN/DNN. Operatively, an operations controller and/or iterator can generate one or more instructions having a calculated shifting bit(s) for communication to the line buffer. Illustratively, the shifting bit(s) can be calculated using various characteristics of the input data as well as the NN/DNN inclusive of the data dimensions. The line buffer can read data for processing, insert the shifting bits and write the data in the line buffer for subsequent processing by cooperating processing unit(s).

What is claimed is: 
     
         1 . A system for enhanced data processing using one or more virtualized hardware iterators in a neural network environment ( 100 ), the system comprising:
 at least one processor ( 205 );   at least one line buffer ( 220 ) operable to perform to read and/or write data; and   at least one memory ( 210 ) in communication with the at least one processor, the at least one memory ( 210 ) having computer-readable instructions stored thereupon that, when executed by the at least one processor, cause the at least one processor to:
 receive one or more initialization parameters from a cooperating controller component ( 235 ) of the neural network environment ( 100 ), the initialization parameters comprising data representative of the dimensions ( 340 ) of the data ( 305 ) to be processed by the neural network environment ( 100 ) and data representative of one or more discontinuities of one or more data elements ( 620 ) between one or more rows of the data ( 305 ); 
 load data ( 305 ) from a cooperating memory component ( 210 ,  225 ) of the neural network environment ( 100 ); 
 calculate shifting bits ( 640 ) representative of the number of bits to shift the one or more data elements ( 620 ) of the data according to the initialization parameters; 
 receive one or more instructions from the cooperating controller component ( 235 ) of the neural network environment to insert one or more shifting bits ( 640 ) into the loaded data ( 610 ) to generate directed line buffer data and to write the directed line buffer data in the at least one line buffer ( 220 ) according to the one or more initialization parameters; and 
 communicate the written data in the at least one line buffer ( 220 ) to the one or more processing components ( 205 ) of the neural network environment ( 100 ) for processing. 
   
     
     
         2 . The system of  claim 1 , wherein the insertion of the one or more shifting bits results in a single write of the directed line buffer data in the at least one line buffer. 
     
     
         3 . The system of  claim 1 , wherein the computer-readable instructions further cause the at least one processor to communicate data that is traversed by a cooperating iterator to the line buffer. 
     
     
         4 . The system of  claim 3 , wherein the computer-readable instructions further cause the at least one processor to traverse the data utilizing one or more sliding windows, the windows operative to select one or more data elements of the data volume as the one or more portions communicated to the one or more processing components. 
     
     
         5 . The system of  claim 4 , wherein the computer-readable instructions further cause the at least one processor to traverse the loaded data using one or more sliding windows that straddle a data dimensional boundary of the loaded data. 
     
     
         6 . The system of  claim 1 , wherein the computer-readable instructions further cause the at least one processor to insert one or more data paddings to the loaded data. 
     
     
         7 . The system of  claim 1 , wherein the computer-readable instructions further cause a copy of the written data in the at least one line buffer to be created and accessed by the one or more processing components of the neural network environment. 
     
     
         8 . A computer-implemented method, comprising:
 receiving one or more initialization parameters from a cooperating controller component ( 235 ) of the neural network environment ( 100 ), the initialization parameters comprising data representative of the dimensions ( 340 ) of the data ( 305 ) to be processed by the neural network environment ( 100 ) and data representative of one or more discontinuities of one or more data elements ( 620 ) between one or more rows of the data ( 305 );   loading data ( 305 ) from a cooperating memory component ( 210 ,  225 ) of the neural network environment ( 100 );   iterating the loaded data according to a selected iteration operation by a cooperating iterator component ( 240 ) of the neural network environment ( 100 );   calculating shifting bits ( 640 ) representative of the number of bits to shift the one or more data elements ( 620 ) of the data according to the initialization parameters;   receiving one or more instructions from the cooperating controller component ( 235 ) of the neural network environment to insert one or more shifting bits ( 640 ) into the loaded data ( 610 ) to generate directed line buffer data and to write the directed line buffer data in a line buffer ( 220 ) according to the one or more initialization parameters; and   communicating the written data in the line buffer ( 220 ) to the one or more processing components ( 205 ) of the neural network environment ( 100 ) for processing.   
     
     
         9 . The computer-implemented method of  claim 8 , wherein the one or more portions of the loaded data are unequal portions. 
     
     
         10 . The computer-implemented method of  claim 8 , wherein the sliding windows are operative to straddle a data dimensional boundary of the data. 
     
     
         11 . The computer-implemented method of  claim 8 , further comprising:
 inserting a padding sub-volume into the loaded data that is defined by the received one or more instructions from the cooperating controller components and by the received one or more initialization parameters.   
     
     
         12 . The computer-implemented method of  claim 8 , further comprising:
 generating a copy of written directed line buffer data.   
     
     
         13 . The computer-implemented method of  claim 12 , further comprising:
 processing the generated copy of the written directed line buffer data by one or more cooperating processing units of the neural network environment.   
     
     
         14 . The computer-implemented method of  claim 8 , further comprising:
 clearing the line buffer of the written directed line buffer data to receive additional directed line buffer data for writing in the line buffer.   
     
     
         15 . The computer-implemented method of  claim 8 , further comprising writing the directed line buffer data in a selected number of lines in the line buffer wherein each line of the line buffer is associated with a cooperating processing unit of the neural network environment. 
     
     
         16 . A computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by one or more processors of a computing device, cause the one or more processors of the computing device to:
 receive one or more initialization parameters from a cooperating controller component ( 235 ) of the neural network environment ( 100 ), the initialization parameters comprising data representative of the dimensions ( 340 ) of the data ( 305 ) to be processed by the neural network environment ( 100 ) and data representative of one or more discontinuities of one or more data elements ( 620 ) between one or more rows of the data ( 305 );   load data ( 305 ) from a cooperating memory component ( 210 ,  225 ) of the neural network environment ( 100 );   iterate the loaded data according to a selected iteration operation by a cooperating iterator component ( 240 ) of the neural network environment ( 100 );   calculate shifting bits ( 640 ) representative of the number of bits to shift the one or more data elements ( 620 ) of the data according to the initialization parameters;   receive one or more instructions from the cooperating controller component ( 235 ) of the neural network environment to insert one or more shifting bits ( 640 ) into the loaded data ( 610 ) to generate directed line buffer data and to write the directed line buffer data in one or more lines of a line buffer ( 220 ) wherein the one or more lines of the line buffer ( 220 ) are associated to one or more processing components ( 205 ) of the neural network environment ( 100 ); and   communicate the written data in the one or more lines of the line buffer to the one or more processing components ( 205 ) of the neural network environment ( 100 ) associated to the one or more lines of the line buffer ( 220 ) for processing.   
     
     
         17 . The computer-readable storage medium of  claim 16 , wherein the instructions further cause the one or more processors of the computing device to:
 insert an additional data volume to the loaded data.   
     
     
         18 . The computer-readable storage medium of  claim 17 , wherein the instructions further cause the one or more processors of the computing device to:
 retrieve a stride value from the initialization parameters representative of the calculated shifting bits.   
     
     
         19 . The computer-readable storage medium of  claim 16 , wherein the instructions further cause the one or more processors of the computing device to:
 process the written directed line buffer data according a selected number of processing cycles wherein the selected number of processing cycles are based on the number of line buffer columns.   
     
     
         20 . The computer-readable storage medium of  claim 16 , wherein the instructions further cause the one or more processors of the computing device to:
 traverse the loaded data utilizing a logical data mapping of the loaded data, the traversing of the loaded data comprising applying one or more sliding windows to the logical data mapping to associate a portion of the loaded data to one or more physical memory addresses.   
     
     
         21 . The computer readable medium of  claim 16 , wherein the memory component cooperates with a physical sensor capable of producing input data comprising audio data, video data, haptic sensory data, and other data for subsequent processing by the one or more cooperating processing units. 
     
     
         22 . The computer readable medium of  claim 21 , wherein the cooperating processing units electronically cooperate with one or more output physical components operative to receive for human interaction processed input data comprising audio data, video data, haptic sensory data and other data. 
     
     
         23 . The computer readable medium of  claim 16 , wherein the calculating a first shifting bit value for the loaded data that is continuous and another shifting bit value for the loaded data that is discontinuous. 
     
     
         24 . The computer readable medium of  claim 16 , further comprising first shifting the loaded according to the calculated first shift bit value, and then shifting the loaded data according to the another shifting bit value.