Accelerating neural networks with low precision-based multiplication and exploiting sparsity in higher order bits

Abstract

An apparatus to facilitate accelerating neural networks with low precision-based multiplication and exploiting sparsity in higher order bits is disclosed. The apparatus includes a processor comprising a re-encoder to re-encode a first input number of signed input numbers represented in a first precision format as part of a machine learning model, the first input number re-encoded into two signed input numbers of a second precision format, wherein the first precision format is a higher precision format than the second precision format. The processor further includes a multiply-add circuit to perform operations in the first precision format using the two signed input numbers of the second precision format; and a sparsity hardware circuit to reduce computing on zero values at the multiply-add circuit, wherein the processor to execute the machine learning model using the re-encoder, the multiply-add circuit, and the sparsity hardware circuit.

1 - 23 . (canceled) 
     
     
         24 . An apparatus for deep learning, comprising:
 a mixed precision unit configured to:
 decompose an element in an input feature map of a deep learning operation into two input elements, the element in the input feature map having a first precision, the two input elements having a second precision that is lower than the first precision, and 
 decompose a weight of the deep learning operation into two weight elements, the weight associated with the element in the input feature map, the weight having the first precision, the two weight elements having the second precision; 
   a multiply-accumulation unit configured to perform a computation using the two input elements and the two weight elements; and   a sparsity unit configured to skip one or more computations of one or more zero-valued elements in the input feature map by the multiply-accumulation unit.   
     
     
         25 . The apparatus of  claim 24 , further comprising:
 a compression unit configured to compress a plurality of elements in the input feature map, the plurality of elements comprising the element.   
     
     
         26 . The apparatus of  claim 25 , wherein the compression unit configured to compress a plurality of weights of the deep learning operation, the plurality of weights comprising the weight. 
     
     
         27 . The apparatus of  claim 24 , wherein multiply-accumulation unit is further configured to:
 generate an output activation of the deep learning operation from the computation.   
     
     
         28 . The apparatus of  claim 27 , wherein the mixed precision unit is further configured to:
 decompose the output activation into two output elements having the second precision.   
     
     
         29 . The apparatus of  claim 27 , further comprising:
 a compression unit configured to compress a plurality of output activations of the deep learning operation, the plurality of output operations comprising the output activation.   
     
     
         30 . The apparatus of  claim 24 , further comprising:
 an additional unit configured to compute the element in the input feature map based on an activation function.   
     
     
         31 . A method for deep learning, comprising:
 decomposing an element in an input feature map of a deep learning operation into two input elements, the element in the input feature map having a first precision, the two input elements having a second precision that is lower than the first precision, the input feature map further comprising one or more zero-valued elements; and   decomposing a weight of the deep learning operation into two weight elements, the weight associated with the element in the input feature map, the weight having the first precision, the two weight elements having the second precision;   performing a computation using the two input elements and the two weight elements; and   skipping one or more computations of the one or more zero-valued elements.   
     
     
         32 . The method of  claim 31 , further comprising:
 compressing the input feature map; and   storing a compressed version of the input feature map in a memory.   
     
     
         33 . The method of  claim 31 , further comprising:
 compressing a plurality of weights of the deep learning operation, the plurality of weights comprising the weight.   
     
     
         34 . The method of  claim 31 , further comprising:
 generating an output activation of the deep learning operation from the computation.   
     
     
         35 . The method of  claim 34 , further comprising:
 decomposing the output activation into two output elements having the second precision.   
     
     
         36 . The method of  claim 34 , further comprising:
 compressing a plurality of output activations of the deep learning operation, the plurality of output operations comprising the output activation.   
     
     
         37 . The method of  claim 31 , further comprising:
 computing the element in the input feature map based on an activation function.   
     
     
         38 . One or more non-transitory computer-readable media storing instructions executable to perform operations, the operations comprising:
 decomposing an element in an input feature map of a deep learning operation into two input elements, the element in the input feature map having a first precision, the two input elements having a second precision that is lower than the first precision, the input feature map further comprising one or more zero-valued elements; and   decomposing a weight of the deep learning operation into two weight elements, the weight associated with the element in the input feature map, the weight having the first precision, the two weight elements having the second precision;   performing a computation using the two input elements and the two weight elements; and   skipping one or more computations of the one or more zero-valued elements.   
     
     
         39 . The one or more non-transitory computer-readable media of  claim 38 , wherein the operations further comprise:
 compressing the input feature map; and   storing a compressed version of the input feature map in a memory.   
     
     
         40 . The one or more non-transitory computer-readable media of  claim 38 , wherein the operations further comprise:
 compressing a plurality of weights of the deep learning operation, the plurality of weights comprising the weight.   
     
     
         41 . The one or more non-transitory computer-readable media of  claim 38 , wherein the operations further comprise, further comprising:
 generating an output activation of the deep learning operation from the computation.   
     
     
         42 . The one or more non-transitory computer-readable media of  claim 41 , wherein the operations further comprise, further comprising:
 decomposing the output activation into two output elements having the second precision.   
     
     
         43 . The one or more non-transitory computer-readable media of  claim 38 , wherein the operations further comprise:
 computing the element in the input feature map based on an activation function.