Efficient Architectures For Deep Learning Algorithms

Abstract

A computer system including a plurality of SIMD engines and a corresponding plurality of output register sets. Operand A register file stores one or more Operand A values, each including a plurality of operand words. Operand B register file stores one or more Operand B values, each including a plurality of operand words. Operand A distribution circuit receives an Operand A value from the Operand A register file, and selectively routes one or more of the operand words of the received Operand A value to create a plurality of input Operand A values, which are selectively routed to the SIMD engines. Operand B distribution circuit receives one or more Operand B values from the Operand B register file, and selectively routes one or more of the operand words of the Operand B value(s) to create a plurality of input Operand B values, which are selectively routed to the SIMD engines.

We claim: 
     
         1 . A computer system comprising:
 a plurality of single instruction, multiple data (SIMD) engines;   a plurality of output register sets, each coupled to a corresponding one of the plurality of SIMD engines;   a first operand register file that stores a first plurality of operand values, wherein each of the first plurality of operand values includes a plurality of operand words;   a second operand register file that stores a second plurality of operand values, wherein each of the second plurality of operand values includes a plurality of operand words;   an first input distribution circuit coupled to receive a first operand value from the first operand register file, wherein the first input distribution circuit selectively routes one or more of the operand words of the first operand value to create a plurality of first input operand values, wherein each of the first input operand values is routed to a corresponding one of the plurality of SIMD engines; and   a second input distribution circuit coupled to receive one or more second operand values from the second operand register file, wherein the second input distribution circuit selectively routes one or more of the operand words of the one or more second operand values to create a plurality of second input operand values, wherein each of the second input operand values is routed to a corresponding one of the plurality of SIMD engines.   
     
     
         2 . The computer system of  claim 1 , wherein the first input distribution circuit selectively routes the first operand value as the first input operand value to each of the plurality of SIMD engines. 
     
     
         3 . The computer system of  claim 1 , wherein the first input distribution circuit selectively routes a single one of the operand words of the first operand value to create all of the first input operand values. 
     
     
         4 . The computer system of  claim 1 , wherein the first input distribution circuit selectively routes the operand words of the first operand value such that each of the first input operand values comprises a repeated operand word of the first operand value. 
     
     
         5 . The computer system of  claim 1 , wherein the second input distribution circuit selectively routes a different second operand value to each of the plurality of SIMD engines. 
     
     
         6 . The computer system of  claim 1 , wherein the second input distribution circuit selectively routes a different second operand value to at least two of the plurality of SIMD engines. 
     
     
         7 . The computer system of  claim 1 , wherein the second input distribution circuit selectively routes the same second operand value to each of the plurality of SIMD engines. 
     
     
         8 . The computer system of  claim 1 , wherein the second input distribution circuit includes a first plurality of second operand buffers, each of the first plurality of second operand buffers configured to store one of the second operand values from the second operand register file. 
     
     
         9 . The computer system of  claim 8 , wherein the second operand register file comprises a plurality of register files, each coupled to a corresponding one of the second operand buffers. 
     
     
         10 . The computer system of  claim 8 , wherein the second input distribution circuit comprises means for coupling the second operand buffers to the plurality of SIMD engines in parallel. 
     
     
         11 . The computer system of  claim 8 , wherein the second input distribution circuit further includes a second plurality of second operand buffers, each of the second plurality of second operand buffers configured to store one of the second operand values from the second operand register file. 
     
     
         12 . The computer system of  claim 1 , wherein the second operand register file comprises a plurality of register files, wherein the plurality of register files are coupled to the second input distribution circuit in parallel. 
     
     
         13 . The computer system of  claim 12 , wherein the second input distribution circuit comprises a shift logic circuit coupled to receive a plurality of the second operand values from the plurality of register files of the second operand register file, wherein the shift logic circuit is configured to shift the operand words of the plurality of the second operand values to create a plurality of input second operand values. 
     
     
         14 . The computer system of  claim 13 , further comprising a plurality of second operand buffers, coupled to receive the plurality of input second operand values from the shift logic circuit. 
     
     
         15 . The computer system of  claim 14 , further comprising switching circuitry coupling the plurality of second operand buffers to the plurality of SIMD engines. ( FIG. 19 ) 
     
     
         16 . The computer system of  claim 1 , further comprising:
 an output circuit including a plurality of output register sets, wherein each of the output register sets is coupled to a corresponding one of the SIMD engines.   
     
     
         17 . The computer system of  claim 16 , wherein each of the output register sets is configured to provide an accumulation value to the corresponding SIMD engine and store an accumulation value provided by the corresponding SIMD engine. 
     
     
         18 . The computer system of  claim 16 , wherein each of the output register sets is independently addressed. 
     
     
         19 . The computer system of  claim 1 , wherein each of the SIMD engines is configured to multiply the operand words of the first input operand value received from the first input distribution circuit with the operand words of the second input operand value received from the second input distribution circuit, whereby each of the SIMD engines generates a corresponding plurality of product values. 
     
     
         20 . The computer system of  claim 19 , further comprising an output circuit including a plurality of output register sets, wherein each of the output register sets is coupled to a corresponding one of the SIMD engines. 
     
     
         21 . The computer system of  claim 20 , wherein each of the output register sets is configured is configured to provide a corresponding plurality of accumulation values to the corresponding one of the SIMD engines, wherein each of the SIMD engines is configured to add the received accumulation values to the corresponding generated product values, whereby each of the SIMD engines generates a corresponding plurality of updated accumulation values. 
     
     
         22 . The computer system of  claim 21 , wherein each of the output register sets is configured to receive and store the corresponding plurality of updated accumulation values from the corresponding one of the SIMD engines. 
     
     
         23 . A method of performing matrix multiplication of a first matrix and a second matrix using a computer system including a plurality of single instruction multiple data (SIMD) engines and a plurality of corresponding output registers, the method comprising:
 identifying a plurality of non-zero entries included in the first matrix, wherein each of the non-zero entries has a corresponding column address and a corresponding row address within the first matrix;   for each non-zero entry of the identified non-zero entries, identifying one of the SIMD engines and a corresponding one of the output registers to process the non-zero entry in response to the corresponding row address of the non-zero entry;   sorting the non-zero entries based on the identified SIMD engines and corresponding output registers, thereby creating a plurality of first operand values, wherein each of the first operand values includes a plurality of the non-zero entries, each having a different identified SIMD engine and corresponding output register; and   routing the first operand values to the SIMD engines to perform multiply operations, wherein the routing causes each of the non-zero entries included in the first operand values to be provided to the identified SIMD engines.   
     
     
         24 . The method of  claim 23 , further comprising:
 for each non-zero entry of the identified non-zero entries, identifying a row of entries within the second matrix in response to the corresponding column address of the non-zero entry; and   routing the identified rows of entries to the SIMD engines to perform multiply operations, wherein the each of the SIMD engines multiples a non-zero entry with its identified row of entries.   
     
     
         25 . The method of  claim 24 , further comprising:
 for each non-zero entry of the identified non-zero entries, identifying a row within the corresponding one of the output registers in response to the corresponding row address of the non-zero entry; and   performing accumulate operations by accessing the identified rows of the output registers.   
     
     
         26 . The method of  claim 23 , wherein each row of the first matrix represents a weight vector in a machine learning system, and each column of the second matrix represents an activation vector in the machine learning system.