Low-voltage, very-low-power conductance mode neuron

Abstract

A neural network including a number of synaptic weighting elements, and a neuron stage; each of the synaptic weighting elements having a respective synaptic input connection supplied with a respective input signal; and the neuron stage having inputs connected to the synaptic weighting elements, and being connected to an output of the neural network supplying a digital output signal. The accumulated weighted inputs are represented as conductances, and a conductance-mode neuron is used to apply nonlinearity and produce an output. The synaptic weighting elements are formed by memory cells programmable to different threshold voltage levels, so that each presents a respective programmable conductance; and the neuron stage provides for measuring conductance on the basis of the current through the memory cells, and for generating a binary output signal on the basis of the total conductance of the synaptic elements.

1. A neural network comprising:
 a plurality of synaptic weighting elements organized in a first set and a second set, each having a respective programmable conductance, each of the synaptic weighting elements having a respective synaptic input connection for receiving an input signal and an output connection;  
 a neuron stage coupled to the synaptic weighting elements;  
 a conductance comparing circuit within the neuron stage, the conductance comparing circuit comparing the conductance of the first set of synaptic weighting elements to the conductance of the second set of synaptic weighting elements and outputting an indication of which set has the higher conductance without determining the conductance values; and  
 a latch stage coupled to the neuron stage to digitize the outputted indication.  
 
     
     
       2. The network according to  claim 1  wherein the synaptic weighting elements include a nonvolatile memory cell having a floating gate and a control gate in a programmable conductance. 
     
     
       3. The network according to  claim 1 , further including:
 an exciting conductive input terminal coupled to the conductance comparing circuit;  
 an inhibiting conductance input terminal coupled to the conductance comparing circuit;  
 a connection between the first set of the synaptic elements and the exciting conductive input terminal; and  
 a connection between the second set of synaptic elements and the inhibiting conductive input terminal, the conductance comparing circuit outputting an indication of the results of the comparison between the exciting synaptic elements and the inhibiting synaptic elements.  
 
     
     
       4. A neural network comprising:
 a plurality of exciting synaptic elements having a programmable conductance;  
 a plurality of inhibiting synaptic elements having a programmable conductance;  
 a neuron stage having a first and second conductance input connected to the exciting and inhibiting synaptic elements, respectively, the neuron stage including a comparing circuit for  providing a comparison of the conductance of the inhibiting synaptic elements and the exciting synaptic elements and generating an output signal indicative of which conductance is greater without determining the value of the conductance of the inhibiting synaptic elements and the exciting synaptic elements; and  
 a latch stage coupled to the neuron stage to digitize the output signal.  
 
     
     
       5. A neural network comprising:
 a plurality of exciting and inhibiting synaptic weighting elements having a respective programmable conductance;  
 a neuron stage coupled to the plurality of exciting and inhibiting synaptic weighting elements, each of the synaptic weighting elements having an associated exciting or inhibiting conductance and an input connection terminal for receiving a respective input signal and an output coupled to the neuron stage, the neuron stage further including a conductance sensing circuit that generates an output signal indicative of which of the total exciting and inhibiting conductances of the plurality of synaptic weighting elements is greater without determining the actual value, collectively and individually, of the exciting and inhibiting conductances; and  
 a latch stage coupled to the neuron stage to digitize the output signal.  
 
     
     
       6. A neural network comprising:
 a plurality of exciting synaptic weighting elements, each having a respective programmable conductance, each of the exciting synaptic weighting elements having output terminals which are connected in common to each other to provide an output which is the sum of the conductances of all the exciting synaptic weighting elements;  
 a plurality of inhibiting synaptic weighting elements, each having a respective programmable conductance, each of the inhibiting synaptic weighting elements having output terminals which are connected in common to each other to provide an output terminal having an output which is the sum of the conductances of all the inhibiting synaptic weighting elements;  
 a neuron stage having an exciting input connected to the output of the plurality of exciting synaptic weighting elements and having an inhibiting input connected to the output of the plurality of the inhibiting synaptic weighting elements; and  
 a conductance comparing stage for  comparing the conductance between the sum of the exciting synaptic weighting elements and the inhibiting synaptic weighting elements without determining the value of the conductances and outputting a signal indicative of which of the sums of the respective pluralities of synaptic weighting elements has the greatest conductance.  
 
     
     
       7. A network comprising:
   a plurality of synaptic weighting elements organized in a first set and a second set, each set having a respective programmable conductance, each of the synaptic weighting elements having a respective synaptic input connection for receiving an input signal and an output connection;        a conductance comparing circuit coupled to the synaptic weighting elements, the conductance comparing circuit comparing the conductance of the first set of synaptic weighting elements to the conductance of the second set of synaptic weighting elements and outputting an indication of which set has the higher conductance without determining the conductance values; and        a latch stage coupled to the conductance comparing circuit to digitize the outputted indication.     
     
     
       8. The network according to  claim 7  wherein the synaptic weighting elements include a nonvolatile memory cell having a floating gate and a control gate in a programmable conductance. 
     
     
       9. The network according to  claim 7 , further comprising:
   an exciting conductive input terminal coupled to the conductance comparing circuit;        an inhibiting conductance input terminal coupled to the conductance comparing circuit;        a connection between the first set of the synaptic elements and the exciting conductive input terminal; and        a connection between the second set of synaptic elements and the inhibiting conductive input terminal, the conductance comparing circuit outputting an indication of the results of the comparison between the exciting synaptic elements and the inhibiting synaptic elements.     
     
     
       10. A network comprising:
   a plurality of exciting synaptic elements having a programmable conductance;        a plurality of inhibiting synaptic elements having a programmable conductance;        a neuron stage having a first and second conductance input connected to the exciting and inhibiting synaptic elements, respectively, the neuron stage including a comparing circuit providing a comparison of the conductance of the inhibiting synaptic elements and the exciting synaptic elements and generating an output signal indicative of which conductance is greater without determining the value of the conductance of the inhibiting synaptic elements and the exciting synaptic elements; and        a latch stage coupled to the neuron stage to digitize the output signal.     
     
     
       11. A network comprising:
   a plurality of exciting and inhibiting synaptic weighting elements having a respective programmable conductance;        a neuron stage coupled to the plurality of exciting and inhibiting synaptic weighting elements, each of the synaptic weighting elements having an associated exciting or inhibiting conductance and an input connection terminal for receiving a respective input signal and an output coupled to the neuron stage, the neuron stage further including a conductance sensing circuit that generates an output signal indicative of which of the total exciting and inhibiting conductances of the plurality of synaptic weighting elements is greater without determining the actual value, collectively and individually, of the exciting and inhibiting conductances; and        a latch stage coupled to the neuron stage to digitize the output signal.     
     
     
       12. A network comprising:
   a plurality of exciting synaptic weighting elements, each having a respective programmable conductance, each of the exciting synaptic weighting elements having output terminals which are connected in common to each other to provide an output which is the sum of the conductances of all the exciting synaptic weighting elements;        a plurality of inhibiting synaptic weighting elements, each having a respective programmable conductance, each of the inhibiting synaptic weighting elements having output terminals which are connected in common to each other to provide an output terminal having an output which is the sum of the conductances of all the inhibiting synaptic weighting elements;        a neuron stage having an exciting input connected to the output of the plurality of exciting synaptic weighting elements and having an inhibiting input connected to the output of the plurality of the inhibiting synaptic weighting elements; and        a conductance comparing stage comparing the conductance between the sum of the exciting synaptic weighting elements and the inhibiting synaptic weighting elements without determining the value of the conductances and outputting a signal indicative of which of the sums of the respective pluralities of synaptic weighting elements has the greatest conductance.     
     
     
       13. An analog circuit, comprising:
   a plurality of synaptic weighting elements organized in a first set and a second set, each set having a respective programmable conductance, each of the synaptic weighting elements having a respective synaptic input connection for receiving an input signal and an output connection;        a conductance comparing circuit coupled to the plurality of synaptic weighting elements, the conductance comparing circuit comparing the conductance of the first set of synaptic weighting elements to the conductance of the second set of synaptic weighting elements and outputting an indication of which set has the higher conductance without determining the conductance values; and        a latch stage coupled to the conductance comparing circuit to digitize the outputted indication.     
     
     
       14. The analog circuit according to  claim 13  wherein the synaptic weighting elements include a nonvolatile memory cell having a floating gate and a control gate in a programmable conductance. 
     
     
       15. The analog circuit according to  claim 13 , further comprising:
   an exciting conductive input terminal coupled to the conductance comparing circuit;        an inhibiting conductance input terminal coupled to the conductance comparing circuit;        a connection between the first set of the synaptic elements and the exciting conductive input terminal; and        a connection between the second set of synaptic elements and the inhibiting conductive input terminal, the conductance comparing circuit outputting an indication of the results of the comparison between the exciting synaptic elements and the inhibiting synaptic elements.     
     
     
       16. An analog circuit, comprising:
   a plurality of exciting synaptic elements having a programmable conductance;        a plurality of inhibiting synaptic elements having a programmable conductance;        a neuron stage having a first and second conductance input connected to the exciting and inhibiting synaptic elements, respectively, the neuron stage including a comparing circuit providing a comparison of the conductance of the inhibiting synaptic elements and the exciting synaptic elements and generating an output signal indicative of which conductance is greater without determining the value of the conductance of the inhibiting synaptic elements and the exciting synaptic elements; and        a latch stage coupled to the neuron stage to digitize the output signal.     
     
     
       17. An analog circuit, comprising:
   a plurality of exciting and inhibiting synaptic weighting elements having a respective programmable conductance;        a conducting sensing circuit coupled to the plurality of exciting and inhibiting synaptic weighting elements, each of the synaptic weighting elements having an associated exciting or inhibiting conductance and an input connection terminal for receiving a respective input signal and an output coupled to the conducting sensing circuit, the conductance sensing circuit that generates an output signal indicative of which of the total exciting and inhibiting conductances of the plurality of synaptic weighting elements is greater without determining the actual value, collectively and individually, of the exciting and inhibiting conductances; and        a latch stage coupled to the conducting sensing circuit to digitize the output signal.     
     
     
       18. An analog circuit, comprising:
   a plurality of exciting synaptic weighting elements, each having a respective programmable conductance, each of the exciting synaptic weighting elements having output terminals which are connected in common to each other to provide an output which is the sum of the conductances of all the exciting synaptic weighting elements;        a plurality of inhibiting synaptic weighting elements, each having a respective programmable conductance, each of the inhibiting synaptic weighting elements having output terminals which are connected in common to each other to provide an output terminal having an output which is the sum of the conductances of all the inhibiting synaptic weighting elements;        a neuron stage having an exciting input connected to the output of the plurality of exciting synaptic weighting elements and having an inhibiting input connected to the output of the plurality of the inhibiting synaptic weighting elements; and        a conductance comparing stage comparing the conductance between the sum of the exciting synaptic weighting elements and the inhibiting synaptic weighting elements without determining the value of the conductances and outputting a signal indicative of which of the sums of the respective pluralities of synaptic weighting elements has the greatest conductance.     
     
     
       19. A system, comprising:
   a plurality of synaptic weighting elements organized in a first set and a second set, each set having a respective programmable conductance, each of the synaptic weighting elements having a respective synaptic input connection for receiving an input signal and an output connection;        a neuron stage coupled to the synaptic weighting elements;        a conductance comparing circuit within the neuron stage, the conductance comparing circuit comparing the conductance of the first set of synaptic weighting elements to the conductance of the second set of synaptic weighting elements and outputting an indication of which set has the higher conductance without determining the conductance values; and        a latch stage coupled to the neuron stage to digitize the outputted indication.     
     
     
       20. The system according to  claim 19  wherein the synaptic weighting elements include a nonvolatile memory cell having a floating gate and a control gate in a programmable conductance. 
     
     
       21. The system according to  claim 19 , further comprising:
   an exciting conductive input terminal coupled to the conductance comparing circuit;        an inhibiting conductance input terminal coupled to the conductance comparing circuit;        a connection between the first set of the synaptic elements and the exciting conductive input terminal; and        a connection between the second set of synaptic elements and the inhibiting conductive input terminal, the conductance comparing circuit outputting an indication of the results of the comparison between the exciting synaptic elements and the inhibiting synaptic elements.     
     
     
       22. A system comprising:
   a plurality of exciting synaptic elements having a programmable conductance;        a plurality of inhibiting synaptic elements having a programmable conductance;        a neuron stage having a first and second conductance input connected to the exciting and inhibiting synaptic elements, respectively, the neuron stage including a comparing circuit providing a comparison of the conductance of the inhibiting synaptic elements and the exciting synaptic elements and generating an output signal indicative of which conductance is greater without determining the value of the conductance of the inhibiting synaptic elements and the exciting synaptic elements; and        a latch stage coupled to the neuron stage to digitize the output signal.     
     
     
       23. A system comprising:
   a plurality of exciting and inhibiting synaptic weighting elements having a respective programmable conductance;        a conducting sensing circuit coupled to the plurality of exciting and inhibiting synaptic weighting elements, each of the synaptic weighting elements having an associated exciting or inhibiting conductance and an input connection terminal for receiving a respective input signal and an output coupled to the conducting sensing circuit, the conductance sensing circuit that generates an output signal indicative of which of the total exciting and inhibiting conductances of the plurality of synaptic weighting elements is greater without determining the actual value, collectively and individually, of the exciting and inhibiting conductances; and        a latch stage coupled to the conducting sensing circuit to digitize the output signal.     
     
     
       24. A system comprising:
   a plurality of exciting synaptic weighting elements, each having a respective programmable conductance, each of the exciting synaptic weighting elements having output terminals which are connected in common to each other to provide an output which is the sum of the conductances of all the exciting synaptic weighting elements;        a plurality of inhibiting synaptic weighting elements, each having a respective programmable conductance, each of the inhibiting synaptic weighting elements having output terminals which are connected in common to each other to provide an output terminal having an output which is the sum of the conductances of all the inhibiting synaptic weighting elements;        a neuron stage having an exciting input connected to the output of the plurality of exciting synaptic weighting elements and having an inhibiting input connected to the output of the plurality of the inhibiting synaptic weighting elements; and        a conductance comparing stage comparing the conductance between the sum of the exciting synaptic weighting elements and the inhibiting synaptic weighting elements without determining the value of the conductances and outputting a signal indicative of which of the sums of the respective pluralities of synaptic weighting elements has the greatest conductance.