Neural processing element for use in a neural network

Abstract

A neural processing element for use in a modular neural network is provided. One embodiment provides a neural network comprising an array of autonomous modules ( 300 ). The modules ( 300 ) can be arranged in a variety of configurations to form neural networks with various topologies, for example, with a hierarchical modular structure. Each module ( 300 ) contains sufficient neurons ( 100 ) to enable it to do useful work as a stand alone system, with the advantage that many modules ( 300 ) can be connected together to create a wide variety of configurations and network sizes. This modular approach results in a scaleable system that meets increased workload with an increase in parallelism and thereby avoids the usually extensive increases in training times associated with unitary implementations.

1 - 23 . (canceled)  
   
   
       24 . A neural network module ( 300 ) comprising an array of neural processing elements ( 100 ) and at least one neural network controller ( 200 ), the neural processing elements comprising: 
 arithmetic logic means ( 50 );    an arithmetic shifter mechanism ( 52 );    data multiplexing means ( 115 , 125 );    memory means ( 56 , 57 , 58 , 59 );    data input means ( 110 ) including at least one input port;    data output means ( 120 ) including at least one output port; and    control logic means ( 54 );    and the controller ( 200 ) comprising    control logic means ( 270 , 280 );    data input means ( 60 ) including at least one input port;    data output means ( 62 ) having at least one output port;    data multiplexing means ( 290 , 292 , 294 );    memory means ( 64 , 68 , 280 );    an address map ( 66 ); and    at least one handshake mechanism ( 210 , 220 , 230 );    characterized in that the controller ( 200 ) is adapted to perform computations on data incoming to and outgoing from the neural processing elements.    
   
   
       25 . The neural network module as claimed in  claim 24  wherein the controller is further adapted to provide addressed and non-addressed instructions to the neural processing elements.  
   
   
       26 . The neural network module as claimed in  claim 24  wherein the memory means of the controller includes programmable memory means.  
   
   
       27 . The neural network module as claimed in  claim 24  wherein the memory means of the controller includes buffer memory associated with said data input means and/or said data output means.  
   
   
       28 . The neural network module as claimed in  claim 24  wherein the controller further comprises a collection of registers and a program counter.  
   
   
       29 . The neural network module ( 300 ) as claimed in  claim 24  wherein the number of processing elements ( 100 ) in the array is a power of two.  
   
   
       30 . A modular neural network comprising: 
 one module ( 300 ) as claimed in  claim 24 , or at least two modules ( 300 ) as claimed in any of  claims 24  to  29  coupled together.    
   
   
       31 . The modular neural network as claimed in  claim 30  further comprising arbitration logic adapted to ensure that during neural activity, only the index from a single module representing the active neuron is output to the processing elements.  
   
   
       32 . The modular neural network as claimed in  claim 31  wherein the arbitration logic is provided on each processing element.  
   
   
       33 . The modular neural network as claimed in  claim 31  wherein the arbitration logic is provided on each module.  
   
   
       34 . The modular neural network as claimed in  claim 31  wherein the arbitration logic comprises a binary tree.  
   
   
       35 . The modular neural network as claimed in  claim 31  wherein the arbitration logic provides a bus grant signal on the output of each processing element.  
   
   
       36 . The modular neural network as claimed in  claim 30  including synchronization means to facilitate data input to the neural network.  
   
   
       37 . The modular neural network as claimed in  claim 36 , wherein said synchronization means enables data to be input only once when the modules ( 300 ) are coupled in hierarchical mode.  
   
   
       38 . The modular neural network as claimed in  claim 36  wherein the synchronization means is adapted to implement a two-line handshake mechanism.  
   
   
       39 . A neural network device comprising a neural network as claimed in  claim 30  wherein an array of processing elements ( 100 ) is implemented on the neural network device with at least one module controller ( 200 ).  
   
   
       40 . The neural network device as claimed in  claim 39 , wherein the device is a field programmable gate array (FPGA) device.  
   
   
       41 . The neural network device as claimed in  claim 39 , comprising one of the following: a full-custom very large scale integration (VLSI) device, a semi-custom VLSI device, or an application specific integrated circuit (ASIC) device.  
   
   
       42 . A neural processing element ( 100 ) for use in a neural network, the processing element comprising: 
 arithmetic logic means ( 50 );    an arithmetic shifter mechanism ( 52 );    data multiplexing means ( 115 , 125 );    memory means ( 56 , 57 , 58 , 59 );    data input means ( 110 ) including at least one input port;    data output means ( 120 ) including at least one output port; and    control logic means ( 54 );    characterized in that the control logic means ( 54 ) is adapted to receive addressed and non-addressed instructions from a module controller.    
   
   
       43 . The neural processing element ( 100 ) as claimed in  claim 42 , wherein each neural processing element ( 100 ) is a single neuron in the neural network.  
   
   
       44 . The neural processing element as claimed in  claim 42  further comprising data bit-size indicating means for enabling operations on different bit-size data values to be executed using the same instruction set.  
   
   
       45 . A neural network controller ( 200 ) for controlling the operation of at least one neural processing element ( 100 ) as claimed in any of  claims 42  to  44 , the controller ( 200 ) comprising: 
 control logic means ( 270 , 280 );    data input means ( 60 ) including at least one input port;    data output means ( 62 ) having at least one output port;    data multiplexing means ( 290 , 292 , 294 );    memory means ( 64 , 68 , 280 );    an address map ( 66 ); and    at least one handshake mechanism ( 210 , 220 , 230 );    characterized in that the controller ( 200 ) is adapted to provide addressed and non-addressed instructions to a neural processing element.    
   
   
       46 . The neural network controller ( 200 ) as claimed in  claim 45  wherein the memory means includes programmable memory means.  
   
   
       47 . The neural network controller ( 200 ) as claimed in  claim 45  wherein the memory means includes buffer memory associated with said data input means and/or said data output means.  
   
   
       48 . A neural network module ( 300 ) comprising an array of neural processing elements ( 100 ) as claimed in  claim 42;  and at least one neural network controller ( 200 ) as claimed in  claim 45 .  
   
   
       49 . A modular neural network comprising: 
 at least one module ( 300 ) comprising an array of neural processing elements ( 100 ) the neural processing elements comprising 
 arithmetic logic means ( 50 );  
 an arithmetic shifter mechanism ( 52 );  
 data multiplexing means ( 115 , 125 );  
 memory means ( 56 , 57 , 58 , 59 );  
 data input means ( 110 ) including at least one input port;  
 data output means ( 120 ) including at least one output port; and  
 control logic means ( 54 );  
   characterized in that the module further comprising arbitration logic adapted to ensure that during neural activity, only the index from a single module representing the active neuron is output to the processing elements.    
   
   
       50 . The modular neural network as claimed in  claim 49  wherein the arbitration logic is provided on each processing element.  
   
   
       51 . The modular neural network as claimed in  claim 49  wherein the arbitration logic is provided on each module.  
   
   
       52 . The modular neural network as claimed in  claim 49  wherein the arbitration logic comprises a binary tree.  
   
   
       53 . The modular neural network as claimed in  claim 49  wherein the arbitration logic provides a bus grant signal on the output of each processing element.  
   
   
       54 . A computer program which upon execution on a computer constitutes together with the computer upon which it is executed an apparatus according to  claim 24 .  
   
   
       55 . A method of classifying data comprising: 
 using the apparatus of  claim 24.