Increasing performance of computational array accelerators

Abstract

To improve performance of a computational array, the architecture of the array can be modified to allow the processing engines of a column to operate in parallel and the clock frequency of the array to be increased. The processing engines of each column of the array can be grouped into a series of row groups. The processing engines of each row group can be loaded with input values, and computations on the input values can be carried out in parallel to generate the column output. One or more flip-flop stages can be inserted into the computational logic of each of the processing engines. The computational logic can then be distributed across the flip-flop stages to reduce the propagation delay between flip-flop stages of the processing engine, hence allowing the clock frequency of the array to be increased.

What is claimed is: 
     
       1. A neural network processor comprising:
 an array of processing engines having a plurality of columns each configured to generate a column sum output, wherein each of the columns includes:
 a plurality of row groups each having a plurality of non-adjacent processing engines along the column, wherein each processing engine of the plurality of non-adjacent processing engines in a row group includes multiplication logic and addition logic distributed across one or more flip-flop stages, each processing engine configured to perform operations including:
 receiving a feature map input value for computing the column sum output; 
 multiplying the feature map input value with a weight value to generate a multiplication result; 
 adding the multiplication result to a partial sum input to generate a partial sum output over a plurality of clock cycles; and 
 outputting the partial sum output from the processing engine to a next row group; and 
 
 a column adder to add partial sum outputs from a last row group to generate the column sum output. 
 
 
     
     
       2. The neural network processor of  claim 1 , wherein the array is configured to be clocked with a clock signal having a clock period being equal to or greater than a maximum propagation delay between flip-flop stages of the addition logic. 
     
     
       3. The neural network processor of  claim 1 , wherein the partial sum output is generated over at least three clock cycles. 
     
     
       4. The neural network processor of  claim 1 , wherein the addition logic is distributed across the one or more flip-flop stages by a synthesis tool. 
     
     
       5. An integrated circuit device comprising:
 an array of processing engines having a plurality of columns each configured to generate a column output, wherein each column in the plurality of columns includes:
 a plurality of row groups each having a plurality of non-adjacent processing engines along the column, wherein each processing engine of the plurality of non-adjacent processing engines in a row group includes a computational circuit and is configured to perform operations including:
 receiving an input value for computing the column output; 
 performing a computation on the input value over a plurality of clock cycles using the computational circuit to derive an intermediate value; and 
 outputting the intermediate value from the processing engine to a next row group; and 
 
 a column accumulator to accumulate intermediate values outputted from a last row group to generate the column output. 
 
 
     
     
       6. The integrated circuit device of  claim 5 , wherein the computation is a multiply-and-accumulate computation. 
     
     
       7. The integrated circuit device of  claim 6 , wherein the column accumulator is an adder circuit. 
     
     
       8. The integrated circuit device of  claim 6 , wherein the multiply-and-accumulate computation is performed over at least three clock cycles. 
     
     
       9. The integrated circuit device of  claim 6 , wherein the input value is a feature map input value, the intermediate value is a partial sum output, and the multiply-and-accumulate computation is performed by multiplying the feature map input value with a weight value to derive a multiplication result, and by adding the multiplication result to a partial sum input to generate the partial sum output. 
     
     
       10. The integrated circuit device of  claim 9 , wherein each processing engine of the plurality of non-adjacent processing engines includes a feature map input buffer, a partial sum input buffer, and a weight input buffer coupled to a weight register. 
     
     
       11. The integrated circuit device of  claim 10 , wherein each processing engine of the plurality of non-adjacent processing engines includes a multiplier circuit coupled to an output of the feature map input buffer and an output of the weight register. 
     
     
       12. The integrated circuit device of  claim 11 , wherein each processing engine of the plurality of non-adjacent processing engines includes an adder circuit coupled to an output of the partial sum input buffer and an output of the multiplier circuit. 
     
     
       13. The integrated circuit device of  claim 5 , wherein the computational circuit includes computational logic distributed across multiple flip-flop stages. 
     
     
       14. The integrated circuit device of  claim 13 , wherein the computational logic includes addition logic that is distributed across the multiple flip-flop stages. 
     
     
       15. The integrated circuit device of  claim 13 , wherein the array of processing engines is configured to be clocked with a clock signal having a clock period being equal to or greater than a maximum propagation delay between the flip-flop stages of the computational circuit. 
     
     
       16. The integrated circuit device of  claim 5 , wherein the plurality of non-adjacent processing engines of a row group are configured to receive their input values belonging to a vector of an input matrix in parallel. 
     
     
       17. The integrated circuit device of  claim 5 , wherein each processing engine of a row group is configured to receive sequential input values on non-adjacent clock cycles. 
     
     
       18. The integrated circuit device of  claim 5 , wherein each processing engine of a row group is configured to output sequential intermediate values to a next row group on non-adjacent clock cycles. 
     
     
       19. A method for computing a column output in a column of processing engines of an array, wherein the column has a series of row groups that each include a plurality of non-adjacent processing engines along the column, the method comprising:
 sequentially performing, by each row group in the series of row groups of the column:
 loading, by each processing engine of the row group, an input value for computing the column output; 
 performing, by each processing engine of the row group, a computation on the input value to derive an intermediate value over a plurality of clock cycles; and 
 outputting, by each processing engine of the row group, the intermediate value to a next row group; and 
 
 accumulating intermediate values from a last row group to generate the column output. 
 
     
     
       20. The method of  claim 19 , wherein sequential input values being loaded into a processing engine are separated by a plurality of clock cycles.