Hardware node having a mixed-signal matrix vector unit

Abstract

Processors and methods for neural network processing are provided. A method in a processor including a matrix vector unit is provided. The method includes receiving vector data and actuation vector data corresponding to at least one layer of a neural network model for processing using the matrix vector unit, where each of digital values corresponding to the vector data and the actuation vector data is represented in a sign magnitude format. The method further includes converting each of the digital values corresponding to at least one of the vector data or the actuation vector data to corresponding analog values and multiplying the vector data and the actuation vector data in an analog domain and providing corresponding multiplication results in a digital domain.

What is claimed: 
     
       1. A method in a processor including a matrix vector unit and at least one multifunction unit, wherein the at least one multifunction unit is coupled to receive at least a subset of output of the matrix vector unit, the method comprising:
 receiving vector data and matrix data for processing using the matrix vector unit; 
 generating a first set of block-floating point (BFP) format values corresponding to the vector data, wherein the first set of BFP format values have a first shared exponent; 
 generating a second set of BFP format values corresponding to the matrix data, wherein the second set of BFP format values have a second shared exponent, and adding the first shared exponent and the second shared exponent to generate a third shared exponent; 
 converting each of the first set of BFP format values corresponding to the vector data to a first set of analog values and converting each of the second set of BFP format values corresponding to the matrix data to a second set of analog values; 
 multiplying the first set of analog values and the second set of analog values in an analog domain to generate analog product values; 
 converting the analog product values into digital product values, and generating BFP result values based on both the digital product values and the third shared exponent; and 
 converting the BFP result values into floating-point format result values and providing at least a subset of the floating-point result values as an input to the at least one multifunction unit for further processing by the at least one multifunction unit, wherein the further processing includes at least one of performing pointwise addition using the at least the subset of the floating-point result values, applying a sigmoid function to the at least the subset of the floating-point result values, or applying a hyperbolic tangent function to the at least the subset of the floating-point result values. 
 
     
     
       2. The method of  claim 1  further comprising:
 providing a first digital value corresponding to at least one of the vector data or the matrix data to a corresponding first digital to analog conversion block to generate a first current value corresponding to the first digital value; 
 providing a second digital value corresponding to at least one of the vector data or the matrix data to a corresponding second digital to analog conversion block to generate a second current value corresponding to the second digital value; and 
 providing a third digital value corresponding to at least one of the vector data or the matrix data to a corresponding third digital to analog conversion block to generate a third current value corresponding to the third digital value. 
 
     
     
       3. The method of  claim 2  further comprising:
 providing the first current value to a first analog multiplier to generate a first voltage corresponding to the first current value; 
 providing the second current value to a second analog multiplier to generate a second voltage corresponding to the second current value; and 
 providing the third current value to a third analog multiplier to generate a third voltage corresponding to the third current value. 
 
     
     
       4. The method of  claim 3 , wherein each of the first analog multiplier, the second analog multiplier, and the third analog multiplier comprises a resistor-ladder network. 
     
     
       5. The method of  claim 3  further comprising:
 providing the first voltage to a first differential pair block and providing the output of the first differential pair block to a single positive wire or a single negative wire based on sign bits associated with corresponding digital values; 
 providing the second voltage to a second differential pair block and providing the output of the second differential pair block to a single positive wire or a single negative wire based on sign bits associated with corresponding digital values; and 
 providing the third voltage to a third differential pair block and providing the output of the third differential pair block to a single positive wire or a single negative wire based on sign bits associated with digital values. 
 
     
     
       6. The method of  claim 5  further comprising converting the differential current carried by the single positive wire and the single negative wire to a differential voltage. 
     
     
       7. The method of  claim 6  further comprising amplifying any differential voltages corresponding to the analog product values and converting the the analog product values into the digital product values using analog to digital converters. 
     
     
       8. A processor comprising a matrix vector unit and at least one multifunction unit, wherein the at least one multifunction unit is coupled to receive at least a subset of output of the matrix vector unit, the processor configured to:
 receive vector data and matrix data corresponding to at least one layer of a neural network model for processing using the matrix vector unit; 
 generate a first set of block-floating point (BFP) format values corresponding to the vector data, wherein the first set of BFP format values have a first shared exponent; 
 generate a second set of BFP format values corresponding to the matrix data, wherein the second set of BFP format values have a second shared exponent, and add the first shared exponent and the second shared exponent to generate a third shared exponent; 
 convert each of the first set of BFP format values corresponding to the vector data to a first set of analog values and converting each of the second set of BFP format values corresponding to the matrix data to a second set of analog values; 
 multiply the first set of analog values and the second set of analog values in an analog domain to generate analog product values; 
 convert the analog product values into digital product values and generate BFP result values based on both the digital product values and the third shared exponent; and 
 convert the BFP result values into floating-point format result values and provide at least a subset of the floating-point result values as an input to the at least one multifunction unit for further processing by the at least one multifunction unit. 
 
     
     
       9. The processor of  claim 8 , wherein the matrix vector unit is further configured to:
 provide a first digital value corresponding to at least one of the vector data or the matrix data to a corresponding first digital to analog conversion block to generate a first current value corresponding to the first digital value; 
 provide a second digital value corresponding to the at least one of the vector data or the matrix data to a corresponding second digital to analog conversion block to generate a second current value corresponding to the second digital value; and 
 provide a third digital value corresponding to the at least one of the vector data or the matrix data to a corresponding third digital to analog conversion block to generate a third current value corresponding to the third digital value. 
 
     
     
       10. The processor of  claim 9 , wherein the matrix vector unit is further configured to:
 provide the first current value to a first analog multiplier to generate a first voltage corresponding to the first current value; 
 provide the second current value to a second analog multiplier to generate a second voltage corresponding to the second current value; and 
 provide the third current value to a third analog multiplier to generate a third voltage corresponding to the third current value. 
 
     
     
       11. The processor of  claim 10 , wherein each of the first analog multiplier, the second analog multiplier, and the third analog multiplier comprises a resistor-ladder network. 
     
     
       12. The processor of  claim 10 , wherein the matrix vector unit is further configured to:
 provide the first voltage to a first differential pair block and provide the output of the first differential pair block to a single positive wire or a single negative wire based on sign bits associated with corresponding digital values; 
 provide the second voltage to a second differential pair block and provide the output of the second differential pair block to a single positive wire or a single negative wire based on sign bits associated with corresponding digital values; and 
 provide the third voltage to a third differential pair block and provide the output of the third differential pair block to a single positive wire or a single negative wire based on sign bits associated with digital values. 
 
     
     
       13. The processor of  claim 12 , wherein the matrix vector unit is further configured to convert the differential current carried by the single positive wire and the single negative wire to a differential voltage. 
     
     
       14. The processor of  claim 13 , wherein the matrix vector unit is further configured to amplify any differential voltages corresponding to the analog product values and convert the the analog product values into the digital product values using analog to digital converters. 
     
     
       15. A system comprising:
 an input message processor configured to process incoming messages; and 
 a neural function unit configured to process the incoming messages, the neural function unit comprising:
 a pipeline configured to process instructions, the pipeline including a matrix vector unit, a first multifunction unit, wherein the first multifunction unit is connected to receive an input from the matrix vector unit, a second multifunction unit, wherein the second multifunction unit is connected to receive an output from the first multifunction unit, and a third multifunction unit, wherein the third multifunction unit is connected to receive an output from the second multifunction unit, and wherein the matrix vector unit is further configured to:
 receive vector data and matrix data corresponding to at least one layer of a neural network model for processing using the matrix vector unit, 
 generate a first set of block-floating point (BFP) format values corresponding to the vector data, wherein the first set of BFP format values have a first shared exponent, 
 generate a second set of BFP format values corresponding to the matrix data, wherein the second set of BFP format values have a second shared exponent, and add the first shared exponent and the second shared exponent to generate a third shared exponent, 
 convert each of the first et of BFP format values corresponding to the vector data to a first set of analog values and converting each of the second set of BFP format values corresponding to the matrix data to a second set of analog values, 
 multiply the first set of analog values and the second set of analog values in an analog domain to generate analog product values, 
 convert the analog product values into digital product values and generate BFP result values based on both the digital product values and the third shared exponent, and 
 convert the BFP result values into floating-point format result values and provide at least a subset of the floating-point result values as an input to the first multifunction unit for further processing by the first multifunction unit. 
 
 
 
     
     
       16. The system of  claim 15 , wherein the matrix vector unit is further configured to:
 provide a first digital value corresponding to at least one of the vector data or the matrix data to a corresponding first digital to analog conversion block to generate a first current value corresponding to the first digital value; 
 provide a second digital value corresponding to at least one of the vector data or the matrix data to a corresponding second digital to analog conversion block to generate a second current value corresponding to the second digital value; and 
 provide a third digital value corresponding to at least one of the vector data or the matrix data to a corresponding third digital to analog conversion block to generate a third current value corresponding to the third digital value. 
 
     
     
       17. The system of  claim 16 , wherein the matrix vector unit further configured to:
 provide the first current value to a first analog multiplier to generate a first voltage corresponding to the first current value; 
 provide the second current value to a second analog multiplier to generate a second voltage corresponding to the second current value; and 
 provide the third current value to a third analog multiplier to generate a third voltage corresponding to the third current value. 
 
     
     
       18. The system of  claim 17 , wherein each of the first analog multiplier, the second analog multiplier, and the third analog multiplier comprises a resistor-ladder network. 
     
     
       19. The system of  claim 17 , wherein the matrix vector unit is further configured to:
 provide the first voltage to a first differential pair block and provide the output of the first differential pair block to a single positive wire or a single negative wire based on sign bits associated with corresponding digital values; 
 provide the second voltage to a second differential pair block and provide the output of the second differential pair block to a single positive wire or a single negative wire based on sign bits associated with corresponding digital values; and 
 provide the third voltage to a third differential pair block and provide the output of the third differential pair block to a single positive wire or a single negative wire based on sign bits associated with digital values. 
 
     
     
       20. The system of  claim 19 , wherein the matrix vector unit is further configured to convert the differential current carried by the single positive wire and the single negative wire to a differential voltage.