Method and apparatus with neural network processing

Abstract

A neural network device includes a shift register circuit, a control circuit, and a processing circuit. The shift register circuit includes registers configured to, in each cycle of cycles, transfer stored data to a next register and store new data received from a previous register to a current register. The control circuit is configured to sequentially input data of input activations included in an input feature map into the shift register circuit in a preset order. The processing circuit, includes crossbar array groups that receive input activations from at least one of the registers and perform a multiply-accumulate (MAC) operation with respect to the received input activation and weights, is configured to accumulate and add at least some operation results output from the crossbar array groups in a preset number of cycles to obtain an output activation in an output feature map.

What is claimed is: 
     
       1. A neural network device comprising:
 a shift register circuit comprising registers configured to, in each cycle of plural cycles, transfer stored data to a next register and store new data received from a previous register; 
 a control circuit configured to sequentially input data of input activations included in an input feature map into the shift register circuit in a preset order; and 
 a processing circuit, comprising crossbar array groups that receive the input activations from at least one of the registers and perform a multiply-accumulate (MAC) operation with respect to the received input activation and weights, configured to select at least some of operation results output from the crossbar array groups at a preset number of cycles to be converted and accumulate and add the at least some operation results using a result of the converted to obtain an output activation in an output feature map. 
 
     
     
       2. The neural network device of  claim 1 , wherein the control circuit is further configured to receive a 1-bit zero mark on each of the plural cycles, and in response to a value of the zero mark being  1 , to control the crossbar array groups to omit a MAC operation with respect to input activations corresponding to the zero mark. 
     
     
       3. The neural network device of  claim 1 , wherein crossbar arrays included in one crossbar array group of the crossbar array groups share a same input activation. 
     
     
       4. The neural network device of  claim 3 , wherein each of the crossbar arrays comprises:
 a plurality of row lines; 
 a plurality of column lines intersecting the plurality of row lines; and 
 memory cells respectively disposed at the intersections of the plurality of row lines and the plurality of column lines, and configured to store the weights included in a weight kernel. 
 
     
     
       5. The neural network device of  claim 3 , wherein the processing circuit is further configured to obtain a first output activation using an operation result output from one of the crossbar arrays, and obtain a second output activation using an operation result output from another of the crossbar arrays. 
     
     
       6. The neural network device of  claim 3 , wherein a number of the crossbar arrays included in the one crossbar array group corresponds to a width of a weight kernel. 
     
     
       7. The neural network device of  claim 1 , wherein a number of registers that transfer input activation to the crossbar array groups from the registers corresponds to a height of a weight kernel. 
     
     
       8. The neural network device of  claim 1 , wherein the processing circuit is further configured to convert the selected operation results into a 2′s complement format, and accumulate and add the converted operation results to obtain the output activation. 
     
     
       9. The neural network device of  claim 1 , wherein
 the processing circuit comprises an output line through which the output activation is output, and 
 the output line corresponds to an output of one of a plurality of layers constituting a neural network, and is directly connected to an input line of a next layer. 
 
     
     
       10. The neural network device of  claim 9 , wherein the next layer comprises either one or both of a convolution layer and a pooling layer. 
     
     
       11. A method of a neural network device, the method comprising:
 sequentially inputting input activations included in an input feature map into a shift register circuit in a preset order; 
 receiving an input activation of the input activations from at least one of a plurality of registers, of the shift register circuit, corresponding to a corresponding crossbar array group of crossbar array groups and performing a multiply-accumulate (MAC) operation on the received input activation and weights; and 
 obtaining an output activation included in an output feature map by selecting at least some of operation results output from the crossbar array groups at a preset number of cycles to be converted and accumulating and adding the at least some of operation results based on a result of the converted. 
 
     
     
       12. The method of  claim 11 , further comprising:
 receiving a 1-bit zero mark on each cycle of the sequentially inputting of the input activations; and 
 in response to the a value of the zero mark being  1 , controlling the crossbar array groups to omit the MAC operation with respect to input activations corresponding to the zero mark. 
 
     
     
       13. The method of  claim 11 , wherein crossbar arrays included in one crossbar array group of the crossbar array groups share a same input activation. 
     
     
       14. The method of  claim 13 , wherein each of the crossbar arrays comprises:
 a plurality of row lines; 
 a plurality of column lines intersecting the plurality of row lines; and 
 memory cells respectively disposed at the intersections of the plurality of row lines and the plurality of column lines, and configured to store the weights of a weight kernel. 
 
     
     
       15. The method of  claim 13 , further comprising:
 obtaining a first output activation using an operation result output from one of the crossbar arrays; and 
 obtaining a second output activation using an operation result output from another crossbar array of the crossbar arrays. 
 
     
     
       16. The method of  claim 13 , wherein a number of the crossbar arrays included in the one crossbar array group corresponds to a width of a weight kernel. 
     
     
       17. The method of  claim 11 , wherein a number of registers that transfer input activation to the crossbar array groups from the plurality of registers corresponds to a height of a weight kernel. 
     
     
       18. The method of  claim 11 , wherein the obtaining the output activation comprises:
 converting the selected operation results into a 2′s complement format; and 
 accumulating and adding the converted operation results. 
 
     
     
       19. The method of  claim 11 , further comprising outputting the output activation via an output line, wherein the output line corresponds to an output of one of a plurality of layers constituting a neural network, and is directly connected to an input line of a next layer. 
     
     
       20. The method of  claim 19 , wherein the next layer comprises either one or both of a convolutional layer and a pooling layer.