Application specific integrated circuit accelerators

Abstract

A tile including circuitry for use with machine learning models, the tile including: a first computational array of cells, in which the computational array of cells is a sub-array of a larger second computational array of cells; local memory coupled to the first computational array of cells; and multiple controllable bus lines, in which a first subset of the multiple controllable bus lines include multiple general purpose controllable bus lines couplable to the local memory.

What is claimed is: 
     
       1. A tile comprising circuitry for use with machine learning models, the tile comprising:
 a first computational array of cells, wherein the first computational array of cells is a sub-array of a larger second computational array of cells; 
 local memory coupled to the first computational array of cells; 
 a multiplexer; 
 a plurality of controllable bus lines, 
 wherein a first subset of the plurality of controllable bus lines comprises a plurality of general purpose controllable bus lines couplable to the local memory, 
 wherein at least two of the general purpose controllable bus lines are coupled to an input of the multiplexer, 
 wherein a second subset of the plurality of controllable bus lines comprises a plurality of partial sum controllable bus lines, 
 wherein the plurality of partial sum controllable bus lines are couplable to the first computational array of cells, 
 wherein the plurality of partial sum controllable bus lines are coupled to another tile of the larger second computational array of cells, and are configured to receive a partial sum from, or transfer a partial sum to, the other tile of the larger second computational array of cells. 
 
     
     
       2. The tile of  claim 1 , wherein the plurality of general purpose controllable bus lines comprises:
 a first group of general purpose controllable bus lines extending along a first dimension of the tile; 
 a second group of general purpose controllable bus lines extending along a second dimension of the tile that is different from the first dimension. 
 
     
     
       3. The tile of  claim 2 , wherein the first group of general purpose controllable bus lines comprises:
 at least one general purpose controllable bus line configured to transfer data in a first direction along the first dimension; and 
 at least one general purpose controllable bus line configured to transfer data in a second opposite direction along the first dimension. 
 
     
     
       4. The tile of  claim 3 , wherein the second group of general purpose controllable bus lines comprises:
 at least one general purpose controllable bus line configured to transfer data in a first direction along the second dimension; and 
 at least one general purpose controllable bus line configured to transfer data in a second opposite direction along the second dimension. 
 
     
     
       5. The tile of  claim 1 ,
 wherein an output of the multiplexer is coupled to the local memory, and. 
 
     
     
       6. The tile of  claim 5 , wherein the tile comprises a register coupled to an input select line of the multiplexer. 
     
     
       7. The tile of  claim 1 , wherein each general purpose controllable bus line comprises a conveyer element. 
     
     
       8. The tile of  claim 7 , wherein the conveyer element is a flip-flop. 
     
     
       9. The tile of  claim 1 , wherein each partial sum controllable bus line of the plurality of partial sum controllable bus lines is further coupled to a vector processing unit. 
     
     
       10. The tile of  claim 1 , wherein the second computational array of cells is a systolic array. 
     
     
       11. The tile of  claim 1 , wherein the local memory is configured to receive weight input data, activation input data, and/or instructions from a circuit element external to the tile. 
     
     
       12. The tile of  claim 11 , wherein the circuit element external to the tile is a communication interface of an application specific integrated circuit. 
     
     
       13. The tile of  claim 12 , wherein the first subset of the plurality of controllable bus lines is also coupled to the circuit element external to the tile such that, during operation of the tile, the weight input data, the activation input data, and/or the instructions are passed along the first subset of the plurality of controllable bus lines to the local memory. 
     
     
       14. The tile of  claim 1 , wherein the first subset of the plurality of controllable bus lines is wired directly to a second tile that encompasses an additional sub-array of the larger second computational array of cells. 
     
     
       15. The tile of  claim 14 , wherein the first subset of the plurality of controllable bus lines skip over a third tile that encompasses yet another sub-array of the larger second computational array of cells. 
     
     
       16. A tile comprising circuitry for use with machine learning models, the tile comprising:
 a first computational array of cells, wherein the first computational array of cells is a sub-array of a larger second computational array of cells; 
 local memory coupled to the first computational array of cells; 
 a multiplexer; 
 a plurality of controllable bus lines, 
 wherein a first subset of the plurality of controllable bus lines comprises a plurality of general purpose controllable bus lines couplable to the local memory, 
 wherein at least two of the general purpose controllable bus lines are coupled to an input of the multiplexer, 
 wherein a second subset of the plurality of controllable bus lines consists of a single partial sum controllable bus line, 
 wherein the single partial sum controllable bus line is configured to receive a sum obtained by the first computational array of cells. 
 
     
     
       17. A method of operating an application specific integrated circuit (ASIC) chip comprising a first tile, the first tile comprising (1) a first computational array of cells that is a sub-array of a larger second computational array of cells, (2) local memory coupled to the first computational array of cells, and (3) a plurality of controllable bus lines, the method comprising:
 loading, from a first subset of the plurality of controllable bus lines, a plurality of weight inputs into the local memory of the first tile, wherein loading the plurality of weight inputs into the local memory of the first tile comprises receiving an input select signal at a multiplexer that is coupled to the first subset of the plurality of controllable bus lines; 
 transferring a first group of the plurality of weight inputs to cells of the first computational array from the local memory; 
 receiving a plurality of activation inputs at cells of the first computational array of cells; and 
 performing a computation with the first computational array of cells, using the first group of the plurality of weight inputs and the plurality of activation inputs, to provide a plurality of outputs. 
 
     
     
       18. The method of  claim 17 , wherein the multiplexer is coupled to an additional subset of controllable bus lines, and to the local memory. 
     
     
       19. The method of  claim 17 , wherein receiving the input select signal at the multiplexer comprises receiving the input select signal from a register coupled to the multiplexer. 
     
     
       20. The method of  claim 19 , comprising:
 receiving the input select signal from a second tile of the ASIC; and 
 storing the input select signal in the register prior to passing the input select signal to the multiplexer. 
 
     
     
       21. The method of  claim 17 , wherein the plurality of activation inputs are received from the local memory. 
     
     
       22. The method of  claim 21 , comprising:
 prior to the plurality of activation inputs being received at the cells of the first computational array of cells, receiving the plurality of activation inputs from the first subset of the controllable bus lines; and 
 loading the plurality of activation inputs from the first subset of the controllable bus lines in the local memory. 
 
     
     
       23. The method of  claim 17 , comprising:
 receiving instructions from the first subset of the controllable bus lines; and 
 loading the instructions from the first subset of the controllable bus lines in the local memory. 
 
     
     
       24. The method of  claim 17 , comprising:
 receiving, at the tile, the plurality of weight inputs, the plurality of activation inputs, and/or instructions, via the first subset of the plurality of controllable bus lines, from a communication interface of the ASIC. 
 
     
     
       25. The method of  claim 17 , comprising:
 receiving, at the first tile, instructions, via the first subset of the plurality of controllable bus lines, from a second tile encompasses an additional sub-array of the larger second computational array of cells. 
 
     
     
       26. The method of  claim 17 , comprising:
 receiving, at the tile, from a second subset of the plurality of controllable bus lines, a first partial sum; 
 adding, from the first computational array of cells, a computational output to the first partial sum to obtain a second partial sum; 
 outputting the second partial sum to the second subset of the plurality of controllable bus lines. 
 
     
     
       27. The method of  claim 26 , comprising passing the second partial sum from the second subset of the plurality of controllable bus lines to a vector processing unit. 
     
     
       28. The method of  claim 26 , wherein the second subset of the plurality of controllable bus lines extend to a second tile that encompasses an additional sub-array of the larger second computational array of cells.