Data replication for accelerator

Abstract

A direct memory access (DMA) engine can be used to multicast data from system memory to a target memory for loading into an array. The DMA engine may include a controller that is configured to receive a data transfer request, and generate a set of write operations for the output interface. The set of write operations can include, for each of multiple partitions of the target memory, a write operation to write usable data from the multicast data to an address offset in the corresponding partition, and an additional write operation to write filler data from the multicast data to a null device address.

What is claimed is: 
     
       1. A computing system comprising:
 system memory; 
 a neural network accelerator including a systolic array and a state buffer configured to store data for loading into the systolic array, wherein the state buffer includes a partition for each row of the systolic array; and 
 a direct memory access (DMA) engine configured to transfer data from the system memory to the state buffer, the DMA engine including:
 an input interface configured to perform read operations according to a source descriptor; 
 a multicast buffer configured to store multicast data read from the system memory via the input interface; 
 a controller configured to:
 determine that a transfer request is for multicasting data to the systolic array; and 
 process a set of destination descriptors, the set of destination descriptors including, for each of a plurality of rows of the systolic array:
 a destination descriptor to write usable data stored in the multicast buffer to a row-dependent address offset in the partition of the state buffer allocated for the corresponding row of the systolic array; and 
 an additional destination descriptor to write unused data from the multicast buffer to a null device address; and 
 
 
 an output interface configured to perform multicast write operations according to the set of destination descriptors. 
 
 
     
     
       2. The computing system of  claim 1 , wherein the computing system is configured to discard a write operation to the null device address, and generate a response to the DMA engine indicating that the write operation to the null device address is complete. 
     
     
       3. The computing system of  claim 1 , wherein the set of destination descriptors includes, for a last row of the systolic array, one destination descriptor to write usable data for the last row from the multicast buffer to a last partition of the state buffer. 
     
     
       4. The computing system of  claim 1 , wherein the row-dependent address offset increases by a length of a datatype of a feature map element for each successive row of the systolic array. 
     
     
       5. A direct memory access engine comprising:
 an input interface configured to read data; 
 an output interface configured to write data to a target memory organized as partitions; 
 a multicast buffer configured to store multicast data from the input interface; 
 and 
 a controller configured to:
 receive a data transfer request; and 
 generate a set of write operations for the output interface, the set of write operations including, for each of a plurality of the partitions of the target memory:
 a write operation to write usable data stored in the multicast buffer to an address offset in the corresponding partition, wherein the address offset increases for each successive partition; and 
 an additional write operation to write filler data stored in the multicast buffer to a null device address. 
 
 
 
     
     
       6. The direct memory access engine of  claim 5 , wherein the data transfer request includes a multicast indicator indicating the data transfer request is a multicast request. 
     
     
       7. The direct memory access engine of  claim 5 , wherein the data transfer request includes a source address and a data length that are used to read the multicast data stored in system memory. 
     
     
       8. The direct memory access engine of  claim 5 , wherein the data transfer request includes a pointer pointing to a set of destination descriptors. 
     
     
       9. The direct memory access engine of  claim 5 , wherein the data transfer request further includes a replication number indicating a number of partitions to write. 
     
     
       10. The direct memory access engine of  claim 5 , wherein the set of write operations includes, for a last partition of the target memory, one write operation to write usable data for the last partition from the multicast buffer to the last partition of the target memory. 
     
     
       11. The direct memory access engine of  claim 5 , wherein the set of write operations are generated according to a set of destination descriptors. 
     
     
       12. The direct memory access engine of  claim 5 , wherein the usable data is feature map data or weight data for a neural network. 
     
     
       13. The direct memory access engine of  claim 5 , wherein the target memory is a state buffer of a neural network accelerator. 
     
     
       14. The direct memory access engine of  claim 13 , wherein the neural network accelerator includes a systolic array, and each partition of the target memory corresponds to a row of the systolic array. 
     
     
       15. The direct memory access engine of  claim 5 , wherein the null device address is memory-mapped in a computing system, and wherein the computing system is configured to drop the write operation to the null device address and generate a response to the direct memory access engine indicating the write operation to the null device address is complete. 
     
     
       16. A computer-implemented method comprising:
 receiving, by a direct memory access (DMA) engine, a data transfer request to transfer data at a source address from system memory to a target memory for loading into an array; 
 reading, by the DMA engine, the data at the source address; 
 storing, by the DMA engine, the data as multicast data in a multicast buffer; 
 processing, by the DMA engine, a set of destination descriptors, the set of destination descriptors including, for each of a plurality of rows of the array:
 a destination descriptor to write usable data stored in the multicast buffer to a row-dependent address offset in a corresponding partition of the target memory; and 
 an additional destination descriptor to write filler data stored in the multicast buffer to a null device address; and 
 
 multicasting, by the DMA engine, the usable data for each row of the array from the multicast buffer to the target memory according to the set of destination descriptors. 
 
     
     
       17. The computer-implemented method of  claim 16 , wherein the null device address is memory-mapped to an address in a computing system that is outside an address range of the system memory. 
     
     
       18. The computer-implemented method of  claim 17 , further comprising:
 issuing a write operation to write the filler data to the null device address; and 
 receiving a response from the computing system indicating the write operation is complete. 
 
     
     
       19. The computer-implemented method of  claim 16 , wherein the usable data corresponds to feature map data or weight data for a neural network. 
     
     
       20. The computer-implemented method of  claim 16 , wherein the array is a systolic array.