Forward-style Gradient GeMMs

Abstract

A method for improving runtime performance and alleviating place and route issues in a reconfigurable computing system includes receiving a compute graph for execution on a reconfigurable dataflow processor. The compute graph includes a node specifying a template-based operation on a first and second tensor having a shared batch dimension B. The node may be split into B nodes. Each of the template-based operations on the pair of tensors may be replace with a GeMM operation on the first reduced rank tensor slice and a tile. B nodes that specify the GeMM operation may be appended with at least one first addition node that accepts input from the B nodes to produce a first modified compute graph. The first modified compute graph may be executed. The method describes a significant improvement to overall compute utilization across gradient-sections. Spatial tiling of tensors facilitates gradient calculation without the use of accumulators.

What is claimed is: 
     
         1 . A system for reducing latency and increasing throughput in reconfigurable dataflow processors, the system comprising:
 a host computer comprising a graph optimization module configured to conduct a method comprising:
 receiving a compute graph for execution on a reconfigurable dataflow processor (RDP), the compute graph comprising a node specifying a template-based gradient operation on a first tensor and a second tensor, the first and second tensors having a shared batch dimension of length B; 
 splitting the node into B nodes that each specify the template-based operation the first and second tensors; 
 replacing each of the template-based operations on the pair of tensors with a general matrix multiplication (GeMM) operation on the first reduced rank tensor slice and a tile to produce B nodes that specify the GeMM operation; 
 appending the B nodes that specify the GeMM operation with at least one first addition (ADDN) node that accepts input from the B nodes that specify the GeMM operation to produce a first modified compute graph; and 
   a reconfigurable dataflow processor (RDP) configured to execute the first modified compute graph.   
     
     
         2 . The system of  claim 1 , further comprising slicing the first and second tensors on the shared batch dimension of length B to produce B first tensor slices and B second tensor slices, each of the first and second tensors of the B first tensor slices and B second tensor slices having a batch dimension of length 1. 
     
     
         3 . The system of  claim 2 , further comprising reshaping each of the first and second tensors of the B first tensor slices and B second tensor slices to produce B reduced rank first tensor slices and B reduced rank second tensor slices, a reduced rank first tensor slice of the B reduced rank first tensor slices corresponding to the reduced rank first tensor slice in the GeMM operations. 
     
     
         4 . The system of  claim 3 , further comprising tiling each reduced rank second tensor slice of the B reduced rank second tensor slices to produce B tile sets, each tile set of the B tile sets having indexed tile elements. The system of  claim 4 , wherein each tile of the B tile sets has an equal block size. 
     
     
         6 . The system of  claim 4 , wherein the GeMM operation is performed on a tile of the tile set of the B a pair of tiles having index-paired tile elements. 
     
     
         7 . The system of  claim 1 , wherein one or more of the B nodes that specify the GeMM operation on the first reduced rank tensor and the tile comprise a section. 
     
     
         8 . The system of  claim 7 , wherein each output sub-tensor from each node of the B nodes of a first section are locally summed by at least one first ADDN node to generate a first partial sum, each output sub-tensor from each node of the B nodes of a second section are locally summed (by at least one ADDN node) to generate a second partial sum, and the first partial sum and the second partial sum are locally summed by a second addition (ADDN) node to generate a first intermediate sum. 
     
     
         9 . The system of  claim 8 , wherein the first partial sum is read directly from Dynamic Random-Access Memory (DRAM) and the second partial sum is read from a pattern memory unit (PMU). 
     
     
         10 . A method for reducing latency and increasing throughput in a reconfigurable computing system, the method comprising:
 receiving a compute graph for execution on a reconfigurable dataflow processor (RDP), the compute graph comprising a node specifying a template-based gradient operation on a first tensor and a second tensor, the first and second tensors having a shared batch dimension of length B;   splitting the node into B nodes that each specify the template-based operation the first and second tensors;   replacing each of the template-based operations on the pair of tensors with a general matrix multiplication (GeMM) operation on the first reduced rank tensor slice and a tile to produce B nodes that specify the GeMM operation;   appending the B nodes that specify the GeMM operation with at least one first addition (ADDN) node that accepts input from the B nodes that specify the GeMM operation to produce a first modified compute graph; and   executing the first modified compute graph on the RDP.   
     
     
         10 . The method of  claim 9 , further comprising slicing the first and second tensors on the shared batch dimension of length B to produce B first tensor slices and B second tensor slices, each of the first and second tensors of the B first tensor slices and B second tensor slices having a batch dimension of length 1. 
     
     
         11 . The method of  claim 10 , further comprising reshaping each of the first and second tensors of the B first tensor slices and B second tensor slices to produce B reduced rank first tensor slices and B reduced rank second tensor slices, a reduced rank first tensor slice of the B reduced rank first tensor slices corresponding to the reduced rank first tensor slice in the GeMM operations. 
     
     
         12 . The method of  claim 11 , further comprising tiling each reduced rank second tensor slice of the B reduced rank second tensor slices to produce B tile sets, each tile set of the B tile sets having indexed tile elements. 
     
     
         13 . The method of  claim 12 , wherein each tile of the B tile sets has an equal block size. 
     
     
         14 . The method of  claim 12 , wherein the GeMM operation is performed on a tile of the tile set of the B a pair of tiles having index-paired tile elements. 
     
     
         15 . The method of  claim 9 , wherein one or more of the B nodes that specify the GeMM operation on the first reduced rank tensor and the tile comprise a section. 
     
     
         16 . The method of  claim 15 , wherein each output sub-tensor from each node of the B nodes of a first section are locally summed by at least one first ADDN node to generate a first partial sum, each output sub-tensor from each node of the B nodes of a second section are locally summed by at least one first ADDN node to generate a second partial sum, and the first partial sum and the second partial sum are locally summed by a second addition (ADDN) node to generate a first intermediate sum. 
     
     
         17 . The method of  claim 16 , wherein the first partial sum is read directly from Dynamic Random-Access Memory (DRAM) and the second partial sum is read from a pattern memory unit (PMU). 
     
     
         18 . A computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, wherein the program instructions are executable by a processor to cause the processor to conduct a method comprising:
 receiving a compute graph for execution on a reconfigurable dataflow processor (RDP), the compute graph comprising a node specifying a template-based gradient operation on a first tensor and a second tensor, the first and second tensors having a shared batch dimension of length B;   splitting the node into B nodes that each specify the template-based operation the first and second tensors;   replacing each of the template-based operations on the pair of tensors with a general matrix multiplication (GeMM) operation on the first reduced rank tensor slice and a tile to produce B nodes that specify the GeMM operation;   appending the B nodes that specify the GeMM operation with at least one first addition (ADDN) node that accepts input from the B nodes that specify the GeMM operation to produce a first modified compute graph; and   executing the first modified compute graph on the RDP.