Vector-by-matrix multiplier modules based on non-volatile 2d and 3d memory arrays

Abstract

Embodiments related to nonvolatile memory devices each having a charge storage, an activation input, a signal input and signal output to output an output signal when the activation input receives an activation signal. The output signal is in a range that is based on a charge stored in the charge storage, the activation signal applied to the activation input, and the input signal received at the signal input. The nonvolatile memory devices are arranged in a two dimensional (XY) layer that has horizontal rows in a first dimension (X) and vertical columns in a second dimension (Y), the activation inputs of memory devices of each row are connected to a same activation input. The memory devices of the columns have signal inputs connected to signal outputs of memory devices in the row above and have signal outputs connected to the signal inputs of memory devices in the row below.

What is claimed is: 
     
         1 . A vector-by-matrix multiplier (VMM) module comprising:
 a three-dimensional (3D) memory matrix of nonvolatile memory devices each having a charge storage, an activation input, a signal input to receive an input signal and a signal output to output an output signal when the activation input receives an activation signal;   the output signal being in a range that is based on a charge stored in the charge storage, the activation signal applied to the activation input, and the input signal received at the signal input;   the nonvolatile memory devices arranged in a plurality of two dimensional (2D) (XY) layers that are vertically disposed along a plurality of (Z) columns, the activation inputs of nonvolatile memory devices along a first dimension (X) of each layer connected to a same activation input (WL), the memory devices of each layer having signal inputs connected to signal outputs of memory devices in the layer above along a third dimension (Z) and having signal outputs connected to the signal inputs of memory devices in the layer below along a third dimension (Z).   
     
     
         2 . The module of  claim 1 , wherein VMM module outputs are signal outputs of a top 2D layer of the memory devices connected together along the second dimension (Y). 
     
     
         3 . The module of  claim 1 , wherein each two-dimensional (2D) (XY) layer that is vertically disposed along a plurality of (Z) columns can be selected by the application of a proper input to the activation inputs (WLs) of that layer and the application of a larger input to the activation inputs (WLs) of other layers. 
     
     
         4 . The module of  claim 1 , further comprising:
 activation circuitry to provide the signals to bias the nonvolatile memory devices in a selected layer in a read mode during operation so there would be no change in the charge storage of the nonvolatile memory devices and so that the nonvolatile memory devices operate in analog mode to have output signals which are dot products of weights stored in the charge storages and VMM module input signals.   
     
     
         5 . The module of  claim 4 , wherein the input activation circuitry converts the VMM module input signals to voltage with a diode-connected nonvolatile memory device to bias the activation inputs (WLs) of the selected layer (called gate-coupled) and bias the signal outputs of a bottom 2D layer of the memory devices connected together along the first dimension (X) with a fixed voltage; or
 wherein the input activation circuitry converts the VMM module input signals to voltage with an operational amplifier with a nonvolatile memory device feedback to bias the signal outputs of a bottom 2D layer of the memory devices connected together along the first dimension (X) (called source-coupled) and bias the activation inputs (WLs) of a nonvolatile memory devices in a selected layer connected together along the first dimension (X) with a fixed voltage;   
     
     
         6 . The module of  claim 1 , wherein when the activation inputs (WLs) of a selected layer receive activation signals, VMM module outputs of the top layer output signals that are the vector-by-matrix multiplication of the weight stored in the nonvolatile memory devices of the selected layer proportional to the charges stored in their charge storages and the input signals received at VMM module inputs. 
     
     
         7 . The module of  claim 1 , wherein module outputs add the signal outputs of the nonvolatile memory devices of rows in the second dimension (Y) of the top layer; and
 wherein the output circuitry subtracts the added signal outputs of every other column of the nonvolatile memory devices of rows in the second dimension (Y) of the top layer from the added signal outputs of the prior column of the nonvolatile memory devices of rows in the second dimension (Y) of that layer.   
     
     
         8 . The module of  claim 1 , wherein the VMM module further comprises programming circuitry to write the charges stored into the charge storages based on parameters of a neural network implemented using the VMM module and trained with a training dataset. 
     
     
         9 . The module of  claim 1 , wherein the nonvolatile memory devices are floating gate transistors (FGTs); the stored charges are floating gate charges; the input signals are drain inputs, the output signals are source outputs and the activation inputs are gate or world-line inputs. 
     
     
         10 . The module of  claim 9 , wherein VMM module outputs are drain terminals of a top 2D layer of the memory devices connected together along bit lines (BLs) along the second dimension (Y). 
     
     
         11 . The module of  claim 1 , further comprising:
 input circuitry to input each of a set of input signals into corresponding ones of a set of same VMM module inputs along the second dimension (Y) of memory devices; and   the output circuitry to connect VMM module signal outputs connected in series of the rows of the memory devices in the first dimension (Y) into neural path outputs of a neural network.   
     
     
         12 . A vector-by-matrix multiplier (VMM) module comprising:
 a two-dimensional (2D) memory matrix of nonvolatile memory devices each having a charge storage, an activation input, a signal input and signal output to output an output signal when the activation input receives an activation signal;   the output signal being in a range that is based on a charge stored in the charge storage, the activation signal applied to the activation input, and the input signal received at the signal input;   the nonvolatile memory devices arranged in a two dimensional (XY) layer that has horizontal rows in a first dimension (X) and vertical columns in a second dimension (Y), the activation inputs of memory devices of each row connected to a same activation input, the memory devices of the columns having signal inputs connected to signal outputs of memory devices in the row above and having signal outputs connected to the signal inputs of memory devices in the row below.   
     
     
         13 . The module of  claim 12 , wherein each row can be selected by the application of proper input to its activation input (WL) and the application of larger input to the activation inputs (WLs) of other rows. 
     
     
         14 . The module of  claim 12 , wherein when the activation input of a row receives an activation signal, the memory devices operate in one of a linear mode or a subthreshold mode to cause the output signals from the memory devices in that row to be dot products of the state and an input signal received at each of the memory devices. 
     
     
         15 . The module of  claim 12 , further comprising:
 activation circuitry to provide the signals to bias the nonvolatile memory devices in a selected row in a read mode during operation so there would be no change in the charge storage of the nonvolatile memory devices and so that the nonvolatile memory devices operate in analog mode to have output signals which are dot products of the weight stored in the charge storage and the VMM module input signals; and   output circuitry to measure the VMM module output signals.   
     
     
         16 . The module of  claim 15 , wherein:
 the VMM module inputs are the different signal inputs of the nonvolatile memory devices in the first row along the first dimension (X) biased by input activation circuitry (called linear mode) and the output circuitry is connected to the signal outputs of the memory devices of the last row connected together along the first dimension (X); or   the VMM module output is a signal inputs of the memory devices of the first row connected together along the first dimension (X), input activation circuitry converts the VMM module input signals to voltage with an operational amplifier with a nonvolatile memory device feedback to bias the signal outputs of a last row of the memory devices connected together along the first dimension (X) (called source-coupled) and bias the activation inputs (WLs) of a nonvolatile memory devices in a selected row connected together along the first dimension (X) with a fixed voltage.   
     
     
         17 . The module of  claim 16 , wherein for the case of source-coupled, the module outputs add the signal inputs of the nonvolatile memory devices of the first row along the first dimension (X). 
     
     
         18 . The module of  claim 16 , wherein for the case of linear mode, the output circuitry subtracts the added signal outputs of every other column of the nonvolatile memory devices of rows in the first dimension (X) of the selected row from the added signal outputs of the prior column of the nonvolatile memory devices of rows in the first dimension (X) of that row. 
     
     
         19 . The module of  claim 12 , wherein the VMM module further comprises programming circuitry to write the charges stored into the charge storage based on the parameters of the neural network implemented based on the VMM module and trained with training dataset. 
     
     
         20 . The module of  claim 12 , wherein the nonvolatile memory devices are floating gate transistors (FGTs); the stored charges are floating gate charges; the input signals are drain inputs, the output signals are source outputs and the activation inputs are gate or world-line inputs.