Spiking neural network simulator for image and video processing

Abstract

Described is system for simulating spiking neural networks for image and video processing. The system processes an image with a spiking neural network simulator having a plurality of inter-connected modules. Each module comprises a plurality of neuron elements. Processing the image further comprises performing a neuron state update for each module, that includes aggregating input spikes and updating neuron membrane potentials, and performing spike propagation for each module, which includes transferring spikes generated in a current time step. Finally, an analysis result is output.

What is claimed is: 
     
       1. A system for simulating spiking neural networks for video and image processing, the system comprising:
 one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when executed, the one or more processors perform operations of: 
 processing an image with a spiking neural network simulator having a plurality of inter-connected modules, each module comprising a plurality of neuron elements and implemented within one or more spiking neural network hardware chips, wherein processing the image further comprises:
 performing a neuron state update for each module that includes aggregating input spikes and updating neuron membrane potentials; and 
 performing spike propagation for each module that includes transferring spikes generated in a current time step; and 
 
 outputting an analysis result, 
 wherein at least one buffer is associated with each module, and the spike propagation comprises updating the at least one buffer of a current module using the at least one buffer of a source module. 
 
     
     
       2. The system as set forth in  claim 1 , wherein processing the image comprises:
 based on a type of a task to be performed on the image, determining whether to perform the processing using one of a matrix having at least four dimensions and a matrix having two or less dimensions; and 
 performing the task on the image using one of the matrix having at least four dimensions and the matrix having two or less dimensions. 
 
     
     
       3. The system as set forth in  claim 2 , wherein the matrix having at least four dimensions denotes synaptic connections and weights between a plurality of neuron elements of a source module and a current module in the plurality of inter-connected modules. 
     
     
       4. The system as set forth in  claim 2 , wherein the matrix having two or less dimensions is configured to perform a convolution. 
     
     
       5. The system as set forth in  claim 3 , wherein processing the image further comprises changing the weights for the matrix having at least four dimensions of at least one of the modules when learning occurs. 
     
     
       6. The system as set forth in  claim 1 , wherein the spike propagation steps between modules located on the same computer environment are performed by assigning pointers. 
     
     
       7. A computer-implemented method for simulating spiking neural networks for video and image processing, comprising an act of:
 causing a data processor to execute instructions stored on a non-transitory memory such that upon execution, the data processor performs operations of: 
 processing an image with a spiking neural network simulator having a plurality of inter-connected modules, each module comprising a plurality of neuron elements and implemented within one or more spiking neural network hardware chips, wherein processing the image further comprises:
 performing a neuron state update for each module that includes aggregating input spikes and updating neuron membrane potentials; and 
 performing spike propagation for each module that includes transferring spikes generated in a current time step; and 
 
 outputting an analysis result, 
 wherein at least one buffer is associated with each module, and the spike propagation comprises updating the at least one buffer of a current module using the at least one buffer of a source module. 
 
     
     
       8. The method as set forth in  claim 7 , wherein processing the image comprises:
 based on a type of a task to be performed on the image, determining whether to perform the processing using one of a matrix having at least four dimensions and a matrix having two or less dimensions; and 
 performing the task on the image using one of the matrix having at least four dimensions and the matrix having two or less dimensions. 
 
     
     
       9. The method as set forth in  claim 8 , wherein the matrix having at least four dimensions denotes synaptic connections and weights between a plurality of neuron elements of a source module and a current module in the plurality of inter-connected modules. 
     
     
       10. The method as set forth in  claim 8 , wherein the matrix having two or less dimensions is configured to perform a convolution. 
     
     
       11. The method as set forth in  claim 9 , wherein processing the image further comprises changing the weights for the matrix having at least four dimensions of at least one of the modules when learning occurs. 
     
     
       12. The method as set forth in  claim 7 , wherein the spike propagation steps between modules located on the same computer environment are performed by assigning pointers. 
     
     
       13. A computer program product for simulating spiking neural networks for video and image processing, the computer program product comprising:
 computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having a processor for causing the processor to perform operations of: 
 processing an image with a spiking neural network simulator having a plurality of inter-connected modules, each module comprising a plurality of neuron elements and implemented within one or more spiking neural network hardware chips, wherein processing the image further comprises:
 performing a neuron state update for each module that includes aggregating input spikes and updating neuron membrane potentials; and 
 performing spike propagation for each module that includes transferring spikes generated in a current time step; and 
 
 outputting an analysis result, 
 wherein at least one buffer is associated with each module, and the spike propagation comprises updating the at least one buffer of a current module using the at least one buffer of a source module. 
 
     
     
       14. The computer program product as set forth in  claim 13 , wherein processing the image comprises:
 based on a type of a task to be performed on the image, determining whether to perform the processing using one of a matrix having at least four dimensions and a matrix having two or less dimensions; and 
 performing the task on the image using one of the matrix having at least four dimensions and the matrix having two or less dimensions. 
 
     
     
       15. The computer program product as set forth in  claim 14 , wherein the matrix having at least four dimensions denotes synaptic connections and weights between a plurality of neuron elements of a source module and a current module in the plurality of inter-connected modules. 
     
     
       16. The computer program product as set forth in  claim 14 , wherein the matrix having two or less dimensions is configured to perform a convolution. 
     
     
       17. The computer program product as set forth in  claim 15 , wherein processing the image further comprises changing the weights for the matrix having at least four dimensions of at least one of the modules when learning occurs. 
     
     
       18. The computer program product as set forth in  claim 13 , wherein the spike propagation steps between modules located on the same computer environment are performed by assigning pointers. 
     
     
       19. The system as set forth in  claim 1 , wherein each module comprises an input layer, one or more middle layers, and an output layer. 
     
     
       20. The method as set forth in  claim 7 , wherein each module comprises an input layer, one or more middle layers, and an output layer. 
     
     
       21. The method as set forth in  claim 13 , wherein each module comprises an input layer, one or more middle layers, and an output layer.