Convolutional neural networks on hardware accelerators

Abstract

A hardware acceleration component is provided for implementing a convolutional neural network. The hardware acceleration component includes an array of N rows and M columns of functional units, an array of N input data buffers configured to store input data, and an array of M weights data buffers configured to store weights data. Each of the N input data buffers is coupled to a corresponding one of the N rows of functional units. Each of the M weights data buffers is coupled to a corresponding one of the M columns of functional units. Each functional unit in a row is configured to receive a same set of input data. Each functional unit in a column is configured to receive a same set of weights data from the weights data buffer coupled to the row. Each of the functional units is configured to perform a convolution of the received input data and the received weights data, and the M columns of functional units are configured to provide M planes of output data.

1 . A hardware acceleration component for implementing a convolutional neural network, the hardware acceleration component configured to receive input data and weights data, and configured to generate M planes of output data, the hardware acceleration component comprising:
 an array of N rows and M columns of functional units;   an array of N input data buffers configured to store the input data, wherein each of the N input data buffers is coupled to a corresponding one of the N rows of functional units; and   an array of M weights data buffers configured to store the weights data, wherein each of the M weights data buffers is coupled to a corresponding one of the M columns of functional units;   wherein:
 each functional unit in a row of functional units configured to receive a same set of input data from the input data buffer coupled to the row; 
 each functional unit in a column of functional units configured to receive a same set of weights data from the weights data buffer coupled to the row; 
 each of the functional units is configured to perform a convolution of the received input data and the received weights data; and 
 the M columns of functional units are configured to provide the M planes of output data. 
   
     
     
         2 . The hardware acceleration component of  claim 1 , wherein the input data comprise a three-dimensional volume of data. 
     
     
         3 . The hardware acceleration component of  claim 1 , wherein the input data comprise image data. 
     
     
         4 . The hardware acceleration component of  claim 1 , wherein the input data are segmented into N slices, each of the N slices comprising a three-dimensional volume of data, and each of the N slices stored in a corresponding one of the N input data buffers. 
     
     
         5 . The hardware acceleration component of  claim 1 , wherein the weights data comprise a three dimensional volume of kernel weights. 
     
     
         6 . The hardware acceleration component of  claim 1 , wherein each of the functional units comprises a multiplier component, an adder component, an accumulation register and a shift register. 
     
     
         7 . The hardware acceleration component of  claim 1 , wherein the hardware acceleration component comprises one or more of a field-programmable gate array device, a massively parallel processor array device, a graphics processing unit, and an application-specific integrated circuit. 
     
     
         8 . The hardware acceleration component of  claim 1 , comprising a hardware acceleration component coupled to a data center server component. 
     
     
         9 . A hardware acceleration component for implementing a multi-layer convolutional neural network, the hardware acceleration component configured to receive input data and weights data, the hardware acceleration component comprising:
 a reconfigurable array of rows and columns of functional units;   an array of first input data buffers configured to store the input data, wherein each of the first input data buffers is coupled to a corresponding one of the rows of functional units;   an array of weights data buffers configured to store the weights data, wherein each of the weights data buffers is coupled to a corresponding one of the columns of functional units;   an array of output data buffers, wherein each of the output data buffers is coupled to a corresponding one of the columns of functional units; and   an array of second input data buffers coupled to the output data buffers, wherein each of the second input data buffers is coupled to a corresponding one of the rows of functional units;   wherein:
 each functional unit in a row of functional units is configured to receive a same set of input data from the first input data buffer or the second input data buffer coupled to the row; 
 each functional unit in a column of functional units is configured to receive a same set of weights data from the weights data buffer coupled to the row; and 
 each of the functional units is configured to perform a convolution of the received input data and the received weights data. 
   
     
     
         10 . The hardware acceleration component of  claim 9 , wherein the columns of functional units provide corresponding planes of output data for a first convolutional layer to the output data buffers. 
     
     
         11 . The hardware acceleration component of  claim 10 , wherein the output data for the first convolutional layer is fed back the array of second input data buffers as input data to a second convolutional layer. 
     
     
         12 . The hardware acceleration component of  claim 9 , further comprising:
 an array of bias components coupled to the array of output data buffers, wherein each of the bias components is configured to add a corresponding bias value to the output data in a corresponding one of the output data buffers;   an array of non-linear function components coupled to the array of bias components, wherein each of the non-linear function components is configured to apply a corresponding non-linear function to the data in a corresponding one of the bias components; and   an array of pooling components coupled to the array of output data buffers and the array of non-linear function components, wherein each of the pooling components is configured to subsample data in the non-linear function components.   
     
     
         13 . The hardware acceleration component of  claim 9 , wherein the input data comprise a three-dimensional volume of data. 
     
     
         14 . The hardware acceleration component of  claim 9 , wherein the input data comprise image data. 
     
     
         15 . The hardware acceleration component of  claim 9 , wherein the input data are segmented into slices, each of the slices comprising a three-dimensional volume of data, and each of the slices stored in a corresponding one of the first set of input data buffers. 
     
     
         16 . The hardware acceleration component of  claim 9 , wherein the weights data comprise a three dimensional volume of kernel weights. 
     
     
         17 . The hardware acceleration component of  claim 9 , wherein the hardware acceleration component comprises one or more of a field-programmable gate array device, a massively parallel processor array device, a graphics processing unit, and an application-specific integrated circuit. 
     
     
         18 . The hardware acceleration component of  claim 9 , comprising a hardware accelerator component coupled to a data center server component. 
     
     
         19 . A method for implementing a multi-layer convolutional neural network, the method comprising:
 providing a hardware acceleration component comprising an array of rows and columns of functional units, each of the functional units configured to perform a convolution operation;   segmenting input data into a plurality of slices of input data;   providing each of the slices of input data to each of the functional units in a corresponding row of functional units;   receiving a plurality of weights data kernels, each weights data kernel comprising weights data;   providing each of the weights data kernels to each of the functional units in a corresponding column of functional units, wherein the columns of functional units provide corresponding planes of output data for a first convolutional layer to the output data buffers; and   feeding back the output data for the first convolutional layer as input data to a second convolutional layer.   
     
     
         20 . The method of  claim 20 , further comprising coupling the hardware acceleration component to a data center server component.