Spiking Neural Network

Abstract

A spiking neural network for classifying input pattern signals, comprising a plurality of spiking neurons implemented in hardware or a combination of hardware and software, and a plurality of synaptic elements interconnecting the spiking neurons to form the network. Each synaptic element is adapted to receive a synaptic input signal and apply a weight to the synaptic input signal to generate a synaptic output signal, the synaptic elements being configurable to adjust the weight applied by each synaptic element, and each of the spiking neurons is adapted to receive one or more of the synaptic output signals from one or more of the synaptic elements, and generate a spatio-temporal spike train output signal in response to the received one or more synaptic output signals.

1 . A spiking neural network for classifying input pattern signals, comprising a plurality of spiking neurons implemented in hardware or a combination of hardware and software, and a plurality of synaptic elements interconnecting the spiking neurons to form the network,
 wherein each synaptic element is adapted to receive a synaptic input signal and apply a weight to the synaptic input signal to generate a synaptic output signal, the synaptic elements being configurable to adjust the weight applied by each synaptic element, and   wherein each of the spiking neurons is adapted to receive one or more of the synaptic output signals from one or more of the synaptic elements, and generate a spatio-temporal spike train output signal in response to the received one or more synaptic output signals,   wherein the spiking neural network comprises a first sub-network comprising a first sub-set of the spiking neurons connected to receive synaptic output signals from a first sub-set of the synaptic elements,   wherein the first sub-network is adapted to generate a sub-network output pattern signal from the first sub-set of spiking neurons, in response to a sub-network input pattern signal applied to the first sub-set of synaptic elements, and   wherein the weights of the first sub-set of synaptic elements are configured by training the sub-network on a training set of sub-network input pattern signals, so that the sub-network output pattern signal is unique for every unique sub-network input pattern signal of the training set, wherein the degree of uniqueness is controllable through operating parameters of the first sub-set of spiking neurons and synaptic elements.   
     
     
         2 . The spiking neural network of  claim 1 , wherein a respective distance between each unique sub-network output pattern signal is larger than a predetermined threshold value, the distance being measured by an output pattern metric. 
     
     
         3 . The spiking neural network of  claim 1 , wherein the spiking neurons and synaptic elements are configured such that respective distances, as measured by an output pattern metric, between two output pattern signals of the sub-network generated in response to two respective different sub-network input pattern signals are maximized for all sub-network input pattern signals of the training set. 
     
     
         4 . The spiking neural network of  claim 3 , wherein each respective distance is maximized until the output pattern signals meet at least a first minimum sensitivity threshold required for distinguishing between features of the input pattern signals. 
     
     
         5 . The spiking neural network of  claim 1 , wherein the weights of the first sub-set of synaptic elements are configured by training the sub-network on a training set of sub-network input pattern signals using a semi-supervised training methodology, by inducing spikes at a desired output neuron at a specific time and at other neurons at a time concomitant with the application of the sub-network input pattern signals, the sub-network is configured to respond with the desired sub-network output pattern signal. 
     
     
         6 . The spiking neural network of  claim 5 , wherein a desired spike is artificially induced at a specific time to obtain a unique response to a certain input pattern. 
     
     
         7 . The spiking neural network of  claim 5 , wherein by inducing a spike artificially at a neuron n of the sub-network at a desired time, the nature of the relationship between neurons in preceding layers of the sub-network and the neuron n is established as causal, anti-causal, or non-causal. 
     
     
         8 . The spiking neural network of  claim 1 , wherein the weights of the first sub-set of synaptic elements are configured using a causal-chain spike-timing-dependent plasticity (CC-STDP) learning rule, which enables identification of causal relationships between desired output neurons and neurons in preceding layers of the sub-network, and causes the weights of intervening synaptic elements along this path of causation to be adjusted. 
     
     
         9 . The spiking neural network of  claim 8 , wherein using the CC-STDP learning rule, the sub-network output pattern signal can be steered towards a different population of the sub-network's output neurons that fired in response to a particular sub-network input pattern signal, and their precise firing times, in order to reach a particular sub-network output pattern signal. 
     
     
         10 . The spiking neural network of  claim 8 , wherein the CC-STDP learning rule adjusts weights of the first sub-set of synaptic elements on the basis of a firing event only if the firing event contributes to firing of neurons in subsequent layers of the sub-network. 
     
     
         11 . The spiking neural network of  claim 8 , wherein the training using the CC-STDP learning rule comprises inducing and/or inhibiting spike generation at neurons of the first sub-set of spiking neurons at specific times. 
     
     
         12 . The spiking neural network of  claim 11 , wherein inducing spike generation in a neuron comprises driving a membrane of the neuron to a voltage that exceeds the firing threshold of the membrane, by means of a bias voltage input, thereby inducing the generation of a spike, or injecting an artificial spike into the neuron's output at the specific time. 
     
     
         13 . The spiking neural network of  claim 11 , wherein inhibiting spike generation in a neuron comprises driving a membrane of the neuron to its refractory voltage, or lowest possible voltage, preventing the generation of a spike, or disabling spike generation at the neuron. 
     
     
         14 - 18 . (canceled) 
     
     
         19 . The spiking neural network of  claim 1 , wherein the network comprises a second sub-network comprising a second sub-set of the spiking neurons connected to receive synaptic outputs from a second sub-set of the synaptic elements,
 wherein the second sub-network is adapted to receive a second sub-network input pattern signal applied to the second sub-set of synaptic elements, and generate a corresponding second sub-network output pattern signal from the second sub-set of neurons, and   wherein the configurations of the second sub-set of synaptic elements are adjusted so that the second sub-network output pattern signal is unique for every unique feature in the second sub-network input pattern signals,   wherein the network comprises a third sub-network comprising a third sub-set of the spiking neurons connected to receive synaptic outputs from a third sub-set of the synaptic elements,   wherein the first and second sub-network output pattern signals are input pattern signals of the third sub-network, and   wherein the configurations of the third sub-set of synaptic elements are adjusted such that the third sub-network output pattern signal is unique for every unique feature in the input pattern signal from both the first and second sub-network and unique combinations of them, such that the features that are present in the input pattern signals from both the first and second sub-network are encoded by the third sub-network.   
     
     
         20 . The spiking neural network of  claim 19 , wherein the synaptic elements of the third sub-network are configured such that the input pattern signals from the first and second sub-network are weighted according to importance of the specific features in the input pattern signals. 
     
     
         21 . The spiking neural network of  claim 1 , wherein the network comprises multiple sub-networks of synaptic elements and spiking neurons, for which the sub-network output pattern signal is unique for every unique feature in the sub-network input pattern signals,
 wherein the network can be divided in multiple layers having a particular sequential order in the network, and   wherein the multiple sub-networks are instantiated in the particular sequential order of the multiple layers each respective sub-network belongs to.   
     
     
         22 - 23 . (canceled) 
     
     
         24 . The spiking neural network of  claim 1 , wherein a set of neurons are arranged in one or multiple template networks of neurons, the neurons of a particular template network of neurons forming a sub-network of the neural network,
 wherein the sub-network output pattern signal is unique for every unique feature in the sub-network input pattern signals, and   wherein each template network in the neural network is instantiated as a pre-trained sub-network that was configured according to claim  17 .   
     
     
         25 . The spiking neural network of  claim 1 , wherein the neural network is configured to take as input one or multiple sampled analog or digital input signals and convert the input signals into a representative set of spiking neural network input pattern signals. 
     
     
         26 . (canceled) 
     
     
         27 . A method for classifying input pattern signals using a spiking neural network comprising a plurality of spiking neurons implemented in hardware or a combination of hardware and software, and a plurality of synaptic elements interconnecting the spiking neurons to form the network,
 wherein each of the synaptic elements is adapted to receive a synaptic input signal and apply a weight to the synaptic input signal to generate a synaptic output signal, the synaptic elements being configurable to adjust the weight applied by each synaptic element, and   wherein each of the spiking neurons is adapted to receive one or more of the synaptic output signals from one or more of the synaptic elements, and generate a spatio-temporal spike train output signal in response to the received one or more synaptic output signals,   the method comprising:   defining a first sub-network of the spiking neural network, the first sub-network comprising a first sub-set of the spiking neurons connected to receive synaptic output signals from a first sub-set of the synaptic elements;   configuring the weights of the first sub-set of synaptic elements by training the sub-network on a training set of sub-network input pattern signals, so that the sub-network output pattern signal is unique for every unique sub-network input pattern signal of the training set, wherein the degree of uniqueness is controllable through operating parameters of the first sub-set of spiking neurons and synaptic elements;   applying a sub-network input pattern signal to a first sub-network of the spiking neural network, the first sub-network comprising a first sub-set of the spiking neurons connected to receive synaptic output signals from a first sub-set of the synaptic elements; and   receiving a sub-network output pattern signal generated by the first sub-set of spiking neurons in response to the sub-network input pattern signal, wherein the output pattern signal identifies one or more features of the input pattern signal.   
     
     
         28 . (canceled) 
     
     
         29 . A memory device comprising a template library stored thereon, the template library comprising information on a configuration of one or multiple template networks of spiking neurons for use as a sub-network of a spiking neural network,
 wherein each template network comprises a set of the spiking neurons implemented in hardware or a combination of hardware and software connected to receive synaptic outputs from a set of the synaptic elements, and   wherein each template network is adapted to receive a template network input pattern signal applied to the set of synaptic elements, and generate a corresponding template network output pattern signal from the set of neurons, and   wherein the configurations of the set of synaptic elements are adjusted by training the template network on a training set of template network input pattern signals, so that the template network output pattern signal is unique for every unique template network input pattern signal of the training set, wherein the degree of uniqueness is controllable through operating parameters of the first sub-set of spiking neurons and synaptic elements,   wherein the training set is used to train the template network to perform a particular task,   such that the sub-network of the spiking neural network can be instantiated on the basis of the information on the configuration of the template network in a pre-trained manner to perform the particular task.   
     
     
         30 . A method of composing a spiking neural network, the method comprising
 obtaining one or multiple template networks or information on the configuration of one or multiple template networks from a template library according to  claim 29 , and   instantiating the one or multiple template networks as a sub-network of the spiking neural network such that the sub-network of the spiking neural network is able to perform the particular task the template network was pre-trained to perform.