Efficient convolutional engine

Abstract

A hardware architecture for implementing a convolutional neural network. Certain ones of the convolver units may be controlled to be active and others may be controlled to be non-active by a controller in order to perform convolution with a striding of greater than or equal to two.

What is claimed is: 
     
       1. An apparatus, comprising:
 a controller; 
 a two-dimensional synchronous shift register comprising a three by four array of data storage elements: 
 
       
         
           
             
               [ 
               
                 
                   
                     
                       d 
                       
                         1 
                         , 
                         1 
                       
                     
                   
                   
                     
                       d 
                       
                         1 
                         , 
                         2 
                       
                     
                   
                   
                     
                       d 
                       
                         1 
                         , 
                         3 
                       
                     
                   
                   
                     
                       d 
                       
                         1 
                         , 
                         4 
                       
                     
                   
                 
                 
                   
                     
                       d 
                       
                         2 
                         , 
                         1 
                       
                     
                   
                   
                     
                       d 
                       
                         2 
                         , 
                         2 
                       
                     
                   
                   
                     
                       d 
                       
                         2 
                         , 
                         3 
                       
                     
                   
                   
                     
                       d 
                       
                         2 
                         , 
                         4 
                       
                     
                   
                 
                 
                   
                     
                       d 
                       
                         3 
                         , 
                         1 
                       
                     
                   
                   
                     
                       d 
                       
                         3 
                         , 
                         2 
                       
                     
                   
                   
                     
                       d 
                       
                         3 
                         , 
                         3 
                       
                     
                   
                   
                     
                       d 
                       
                         3 
                         , 
                         4 
                       
                     
                   
                 
               
               ] 
             
           
         
         wherein: 
         data storage element d 1,1  is electrically coupled to data storage element d 2,1 , 
         data storage element d 2,1  is electrically coupled to data storage element d 3,1 , 
         data storage element d 1,2  is electrically coupled to data storage element d 2,2 , 
         data storage element d 2,2  is electrically coupled to data storage element d 3,2 , 
         data storage element d 1,3  is electrically coupled to data storage element d 2,3 , 
         data storage element d 2,3  is electrically coupled to data storage element d 3,3 , 
         data storage element d 1,4  is electrically coupled to data storage element d 2,4 , and 
         data storage element d 2,4  is electrically coupled to data storage element d 3,4 , 
         a first convolver unit comprising a first plurality of multipliers, m 1,1   1 , m 1,2   1 , m 1,3   1 , m 2,1   1 , m 2,2   1 , m 2,3   1 , m 3,1   1 , m 3,2   1 , and m 3,3   1 , wherein:
 multiplier m 1,1   1  is electrically coupled in a permanent manner to data storage element d 1,1 , 
 multiplier m 1,2   1  is electrically coupled in a permanent manner to data storage element d 1,2 , 
 multiplier m 1,3   1  is electrically coupled in a permanent manner to data storage element d 1,3 , 
 multiplier m 2,1   1  is electrically coupled in a permanent manner to data storage element d 2,1 , 
 multiplier m 2,2   1  is electrically coupled in a permanent manner to data storage element d 2,2 , 
 multiplier m 2,3   1  is electrically coupled in a permanent manner to data storage element d 2,3 , 
 multiplier m 3,1   1  is electrically coupled in a permanent manner to data storage element d 3,1 , 
 multiplier m 3,2   1  is electrically coupled in a permanent manner to data storage element d 3,2 , and 
 multiplier m 3,3   1  is electrically coupled in a permanent manner to data storage element d 3,3 ; and 
 
         a second convolver unit comprising a second plurality of multipliers, m 1,1   2 , m 1,2   2 , m 1,3   2 , m 2,1   2 , m 2,2   2 , m 2,3   2 , m 3,1   2 , m 3,2   2 , and m 3,3   2 , wherein:
 multiplier m 1,1   2  is electrically coupled in a permanent manner to data storage element d 1,2 , 
 multiplier m 1,2   2  is electrically coupled in a permanent manner to data storage element d 1,3 , 
 multiplier m 1,3   2  is electrically coupled in a permanent manner to data storage element d 1,4 , 
 multiplier m 2,1   2  is electrically coupled in a permanent manner to data storage element d 2,2 , 
 multiplier m 2,2   2  is electrically coupled in a permanent manner to data storage element d 2,3 , 
 multiplier m 2,3   2  is electrically coupled in a permanent manner to data storage element d 2,4 , 
 multiplier m 3,1   2  is electrically coupled in a permanent manner to data storage element d 3,2 , 
 multiplier m 3,2   2  is electrically coupled in a permanent manner to data storage element d 3,3 , and 
 multiplier m 3,3   2  is electrically coupled in a permanent manner to data storage element d 3,4 , 
 
         wherein, at a start of a first clock cycle:
 data storage element d 1,1  is configured to store data value x 1,1 , 
 data storage element d 1,2  is configured to store data value x 1,2 , 
 data storage element d 1,3  is configured to store data value x 1,3 , 
 data storage element d 1,4  is configured to store data value x 1,4 , 
 data storage element d 2,1  is configured to store data value x 2,1 , 
 data storage element d 2,2  is configured to store data value x 2,2 , 
 data storage element d 2,3  is configured to store data value x 2,3 , 
 data storage element d 2,4  is configured to store data value x 2,4 , 
 data storage element d 3,1  is configured to store data value x 3,1 , 
 data storage element d 3,2  is configured to store data value x 3,2 , 
 data storage element d 3,3  is configured to store data value x 3,3 , and 
 data storage element d 3,4  is configured to store data value x 3,4 , 
 
         wherein, during the first clock cycle, the first convolver unit is controlled by the controller to be active causing:
 multiplier m 1,1   1 , to multiply data value x 1,1  with weight w 1  so as to generate a product w 1 x 1,1 , 
 multiplier m 1,2   1  to multiply data value x 1,2  with weight w 2  so as to generate a product w 2 x 1,2 , 
 multiplier m 1,3   1  to multiply data value x 1,3  with weight w 3  so as to generate a product w 3 x 1,3 , 
 multiplier m 2,1   1  to multiply data value x 2,1  with weight w 4  so as to generate a product w 4 x 2,1 , 
 multiplier m 2,2   1  to multiply data value x 2,2  with weight w 5  so as to generate a product w 5 x 2,2 , 
 multiplier m 2,3   1  to multiply data value x 2,3  with weight w 6  so as to generate a product w 6 x 2,3 , 
 multiplier m 3,1   1  to multiply data value x 3,1  with weight w 7  so as to generate a product w 7 x 3,1 , 
 multiplier m 3,2   1  to multiply data value x 3,2  with weight w 8  so as to generate a product w 8 x 3,2 , 
 multiplier m 3,3   1  to multiply data value x 3,3  with weight w 9  so as to generate a product w 9 x 3,3 , and 
 an adder to compute a first sum of terms including at least the product w 1 x 1,1 , the product w 2 x 1,2 , the product w 3 x 1,3 , the product w 4 x 2,1 , the product w 5 x 2,2 , the product w 6 x 2,3 , the product w 7 x 3,1 , the product w 8 x 3,2 , and the product w 9 x 3,3 , and 
 
         wherein, during the first clock cycle, the second convolver unit is controlled by the controller to be non-active. 
       
     
     
       2. The apparatus of  claim 1 , wherein the first sum of terms additionally includes b 1 , wherein b 1  is a bias value.