Performing concurrent operations in a processing element

Abstract

A processing element (PE) of a systolic array can perform neural networks computations on two or more data elements of an input data set using the same weight. Thus, two or more output data elements corresponding to an output data set may be generated. Based on the size of the input data set and an input data type, the systolic array can process a single data element or multiple data elements in parallel.

What is claimed is: 
     
         1 . An apparatus comprising:
 a processing element (PE) for neural network computations, the PE comprising:
 a first input register configured to store a first Xin element; 
 a second input register configured to store a second Xin element; 
 a weight register configured to store a weight value; 
 a first multiplier configured to receive the first Xin element from the first input register and the weight value from the weight register, and to multiply the first Xin element with the weight value to generate a first multiplication result; and 
 a second multiplier configured to receive the second Xin element from the second input register and the weight value from the weight register, and to multiply the second Xin element with the weight value to generate a second multiplication result. 
   
     
     
         2 . The apparatus of  claim 1 , wherein the PE further comprises:
 a first adder configured to receive the first multiplication result and generate a first Yout element based on the first multiplication result; and   a second adder configured to receive the second multiplication result and generate a second Yout element based on the second multiplication result.   
     
     
         3 . The apparatus of  claim 2 , wherein:
 the first adder is further configured to receive a first Yin element and to generate the first Yout element by adding the first Yin element with the first multiplication result; and   the second adder is further configured to receive a second Yin element and to generate the second Yout element by adding the second Yin element with the second multiplication result.   
     
     
         4 . The apparatus of  claim 3 , wherein:
 the first multiplier and the first adder form a first fused multiplier adder (FMA); and   the second multiplier and the second adder form a second FMA.   
     
     
         5 . The apparatus of  claim 3 , wherein the PE further comprises:
 a first input port to receive the first Yin element; and   a second input port to receive the second Yin element.   
     
     
         6 . The apparatus of  claim 3 , wherein the PE further comprises:
 a first output port to output the first Yout element; and   a second output port to output the second Y out element.   
     
     
         7 . The apparatus of  claim 1 , wherein the PE further comprises:
 a first input port to receive the first Xin element; and   a second input port to receive the second Xin element.   
     
     
         8 . The apparatus of  claim 1 , wherein the PE further comprises:
 a first output port to output the first Xin element as a first Xout element; and   a second output port to output the second Xin element as a second Xout element.   
     
     
         9 . The apparatus of  claim 1 , wherein the first Xin element and the second Xin element are values having fewer bits than the first multiplication result and the second multiplication result. 
     
     
         10 . The apparatus of  claim 9 , wherein the first Xin element and the second Xin element are 8-bit values and the first multiplication result and the second multiplication result are 16-bit values. 
     
     
         11 . A method comprising:
 storing a first Xin element at a first input register of a processing element (PE);   storing a second Xin element at a second input register of the PE;   storing a weight value at a weight register of the PE;   receiving the first Xin element from the first input register and the weight value from the weight register at a first multiplier of the PE;   multiplying the first Xin element with the weight value at the first multiplier to generate a first multiplication result;   receiving the second Xin element from the second input register and the weight value from the weight register at a second multiplier of the PE; and   multiplying the second Xin element with the weight value at the second multiplier to generate a second multiplication result.   
     
     
         12 . The method of  claim 11 , further comprising:
 receiving the first multiplication result at a first adder of the PE;   generating a first Yout element at the first adder based on the first multiplication result;   receiving the second multiplication result at a second adder of the PE; and   generating a second Yout element at the second adder based on the second multiplication result.   
     
     
         13 . The method of  claim 12 , wherein:
 the first adder further receives a first Yin element and generates the first Yout element by adding the first Yin element with the first multiplication result; and   the second adder further receives a second Yin element and generates the second Yout element by adding the second Yin element with the second multiplication result.   
     
     
         14 . The method of  claim 13 , wherein:
 the first Yin element is received via a first input port of the PE; and   the second Yin element is received via a second input port of the PE.   
     
     
         15 . The method of  claim 13 , further comprising:
 outputting the first Yout element via a first output port of the PE; and   outputting the second Yout element via a second output port of the PE.   
     
     
         16 . The method of  claim 11 , further comprising:
 receiving the first Xin element via a first input port of the PE; and   receiving the second Xin element via a second input port of the PE.   
     
     
         17 . The method of  claim 11 , further comprising:
 outputting the first Xin element as a first Xout element at a first output port of the PE; and   outputting the second Xin element as a second Xout element at a second output port of the PE.   
     
     
         18 . The method of  claim 11 , wherein the first Xin element and the second Xin element are values having fewer bits than the first multiplication result and the second multiplication result. 
     
     
         19 . The method of  claim 18 , wherein the first Xin element and the second Xin element are 8-bit values and the first multiplication result and the second multiplication result are 16-bit values. 
     
     
         20 . A circuit for neural network computations, comprising:
 a two-dimensional array comprising processing elements (PEs), wherein each of the PEs comprises:
 a first input register configured to store a first Xin element; 
 a second input register configured to store a second Xin element; 
 a weight register configured to store a weight value; 
 a first multiplier configured to receive the first Xin element from the first input register and the weight value from the weight register, and to multiply the first Xin element with the weight value to generate a first multiplication result; and 
 a second multiplier configured to receive the second Xin element from the second input register and the weight value from the weight register, and to multiply the second Xin element with the weight value to generate a second multiplication result.