Neural Network Processor

Abstract

A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.

What is claimed is: 
     
         1 . (canceled) 
     
     
         2 . A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising:
 a matrix computation unit configured to compute multiplications using a plurality of cells of the matrix computation unit to generate accumulated values, wherein each of the multiplications is between a weight for a neural network layer and an input to the neural network layer; and   a vector computation unit configured to (i) generate a plurality of activated values for the neural network layer based on the accumulated values generated by the matrix computation unit and (ii) generate an output for the neural network layer based on the plurality of activated values.   
     
     
         3 . The circuit of  claim 2 , wherein:
 the matrix computation unit is configured as a multi-dimensional systolic array; and   the plurality of cells are arranged along at least a first dimension and a second dimension of the systolic array, the first dimension being different than the second dimension.   
     
     
         4 . The circuit of  claim 3 , wherein the matrix computation unit is configured to:
 load one or more weights for the neural network layer into the plurality of cells of the matrix computation unit; and   shift one or more weights through the plurality of cells of the matrix computation unit.   
     
     
         5 . The circuit of  claim 4 , wherein the matrix computation unit is configured to:
 shift one or more inputs for the neural network layer into the plurality of cells of the matrix computation unit to perform the multiplications between the weight for the neural network layer and the input to the neural network layer.   
     
     
         6 . The circuit of  claim 4 , wherein the vector computation unit is configured to:
 apply an activation function to each of the accumulated values provided by the matrix computation unit; and   generate the plurality of activated values for the neural network layer based on the activation function that is applied to each of the accumulated values.   
     
     
         7 . The circuit of  claim 4 , wherein each activated value of the plurality of activated values represents an activation input to a second neural network layer and the matrix computation unit is configured to:
 shift one or more weights for the second neural network layer through a first plurality of cells of the systolic array along a first, column dimension of the systolic array; and   shift a plurality of activation inputs to the second neural network layer through a second plurality of cells of the systolic array along a second, row dimension of the systolic array.   
     
     
         8 . The circuit of  claim 7 , wherein the matrix computation unit is configured to:
 generate a vector of accumulated values for the second neural network layer based on dot products of multiplications between the one or more weights for the second neural network layer and different activation inputs to the second neural network layer that are shifted along the second, row dimension of the of the systolic array.   
     
     
         9 . The circuit of  claim 8 , wherein the vector computation unit is configured to:
 receive the vector of accumulated values generated by the matrix computation unit; and   generate a vector of activation values in response to applying an activation function to each accumulated value in the vector of accumulated values.   
     
     
         10 . The circuit of  claim 2 , wherein:
 the matrix computation unit is configured as a multi-dimensional systolic array;   the systolic array includes the plurality of cells being arranged along at least a first dimension of the systolic array and a second dimension of the systolic array; and   the first dimension and the second dimension are the same dimension.   
     
     
         11 . A method for performing neural network computations using a circuit configured to implement a neural network comprising a plurality of neural network layers, the method comprising:
 receiving, by a matrix computation unit in the circuit, a plurality of weights for a neural network layer and inputs to the neural network layer;   computing, using a plurality of cells of a matrix computation unit, multiplications between a weight for the neural network layer and one or more of the inputs to the neural network layer; and   generating, using activation circuity of a vector computation unit in the circuit, an output for the neural network layer based on the multiplications.   
     
     
         12 . The method of  claim 11 , wherein generating the output for the neural network layer comprises:
 generating, by the matrix computation unit, accumulated values for the neural network layer based on the multiplications; and   generating, by the vector computation unit, a plurality of activated values for the neural network layer based on the accumulated values generated by the matrix computation unit.   
     
     
         13 . The method of  claim 12 , wherein computing the multiplications between the weight for the neural network layer and one or more of the inputs comprises:
 computing the multiplications using a multi-dimensional systolic array of the matrix computation unit, and   wherein the plurality of cells are arranged along at least a first dimension and a second dimension of the systolic array, the first dimension being different than the second dimension.   
     
     
         14 . The method of  claim 13 , wherein receiving the plurality of weights for the neural network layer comprises:
 loading one or more weights for the neural network layer into a distinct cell of the plurality of cells of the matrix computation unit; and   shifting one or more weights through one or more cells of the plurality of cells of the matrix computation unit.   
     
     
         15 . The method of  claim 14 , wherein receiving the inputs to the neural network layer comprises:
 shifting one or more inputs for the neural network layer into the distinct cell of the plurality of cells of the matrix computation unit to perform the multiplications between the weight for the neural network layer and one or more of the inputs to the neural network layer.   
     
     
         16 . The method of  claim 12 , wherein generating the plurality of activated values for the neural network layer comprises:
 applying, by the activation circuity of the vector computation unit, an activation function to each of the accumulated values generated by the matrix computation unit; and   generating the plurality of activated values for the neural network layer based on the activation function applied to each of the accumulated values.   
     
     
         17 . The method of  claim 16 , comprising:
 receiving, by the vector computation unit, a vector of accumulated values generated by the matrix computation unit; and   generating, by the vector computation unit, a vector of activation values in response to applying the activation function to each accumulated value in the vector of accumulated values.   
     
     
         18 . The method of  claim 13 , wherein each activated value of the plurality of activated values represents an activation input to a second neural network layer and the method comprises:
 shifting one or more weights for the second neural network layer through a first plurality of cells of the systolic array along a first, column dimension of the systolic array; and   shifting a plurality of activation inputs to the second neural network layer through a second plurality of cells of the systolic array along a second, row dimension of the systolic array.   
     
     
         19 . The method of  claim 18 , comprising:
 generating a vector of accumulated values for the second neural network layer based on dot products of multiplications between the one or more weights for the second neural network layer and different activation inputs provided to the second neural network layer that are shifted along the second, row dimension of the of the systolic array.   
     
     
         20 . The method of  claim 12 , wherein:
 the matrix computation unit is configured as a multi-dimensional systolic array;   the systolic array includes the plurality of cells being arranged along at least a first dimension of the systolic array and a second dimension of the systolic array; and   the first dimension and the second dimension are the same dimension.   
     
     
         21 . One or more non-transitory machine-readable storage devices for storing instructions that are executable by one or more processing devices to cause performance of operations for performing neural network computations using a circuit configured to implement a neural network comprising a plurality of neural network layers, the operations comprising:
 receiving, by a matrix computation unit in the circuit, a plurality of weights for a neural network layer and inputs to the neural network layer;   computing, using a plurality of cells of the matrix computation unit, multiplications between a weight for the neural network layer and one or more of the inputs to the neural network layer; and   generating, using activation circuity of a vector computation unit in the circuit, an output for the neural network layer based on the multiplications.