Learning to classify complex patterns using a VLSI network of spiking neurons

Srinjoy Mitra , Giacomo Indiveri and Stefano Fusi    Institute of Neuroinformatics, UNI|ETH, Zurich  Center for Theoretical Neuroscience, Columbia University, New York srinjoy|giacomo|fusi@ini.phys.ethz.ch

Abstract
Real time classification of complex patterns of trains of spikes is a difficult and important computational problem. Here we propose a compact, low power, fully analog neuromorphic device which can learn to classify complex patterns of mean firing rates. The chip implements a network of integrate-and-fire neurons connected by bistable plastic synapses. Learning is supervised by a teacher which simply provides an extra input to the output neurons during training. The synapses are modified only as long as the current generated by the plastic synapses does not match the output desired by the teacher (as in the perceptron learning rule). Our device has been designed to be able to learn linearly separable patterns and we show in a series of tests that it can classify uncorrelated random spatial patterns of mean firing rates.

1

Introduction

Spike driven synaptic plasticity mechanisms have been thoroughly investigated in recent years to solve two important problems of learning: 1) how to modify the synapses in order to generate new memories 2) how to protect old memories against the passage of time, the ongoing activity and the overwriting of new memories. Temporal patterns of spikes can be encoded with spike-timing dependent plasticity (STDP) mechanisms (e.g. see [1, 2]). However, STDP in its simplest form is not suitable for learning patterns of mean firing rates [3], and most of the STDP learning algorithms proposed solved the problems of memory encoding and memory preservation only for relatively simple patterns of mean firing rates. Recently a new model of stochastic spike-driven synaptic plasticity has been proposed [4] that is very effective in protecting old learned memories, and captures the rich phenomenology observed in neurophysiological experiments on synaptic plasticity, including STDP protocols. It has been shown that networks of spiking neurons that use this synaptic plasticity model can learn to classify complex patterns of spike trains ranging from stimuli generated by auditory/vision sensors to images of characters from the MNIST database [4]. Here we describe a neuromorphic VLSI implementation of this spike-driven synaptic plasticity model and present classification experiments on synthetic data that validate the model's implementation. The silicon neurons and synapses inside the chip are implemented using full custom analog circuits. While the network's spikes are received in input and transmitted in output using asynchronous digital circuits. Each spike is represented as an Address-Event, where the address encodes either the source neuron or the destination synapse. This device is part of an increasing collection of spike-based computing chips that have been recently developed within the framework of Address-Event Representation (AER) systems [5, 6]. There are even multiple implementations of the same spike-driven plasticity model being investigated in parallel [7, 8]. The focus of this paper is to show that the VLSI device proposed here can successfully classify complex patterns of spike trains, producing results that are in accordance with the theoretical predictions. 1


Figure 1: Layout of a test chip comprising a network of I&F neurons and plastic synapses. The placement of a single neuron along with its synapses is highlighted in the top part of the figure. Other highlighted circuits are described in the test. In Section 2 we describe the main features of the spike-based plasticity model and show how they are well suited for future scaled CMOS VLSI technologies; in Section 3 we characterize the functionality of the spike-based learning circuits; in Section 4 we show control experiments on the learning properties of the VLSI network; and in Section 5 we present experimental results on complex patterns of mean firing rates. In Section 6 we present the concluding remarks and point out future outlooks and potential applications of this system.

2

Implementation of the spike-based plasticity mechanism

Physical implementations of long lasting memories, either biological or electronic, are confronted with two hard limits: the synaptic weights are bounded (they cannot grow indefinitely or become negative), and the resolution of the synapse is limited (i.e. the synaptic weight cannot have an infinite number of states). These constraints, usually ignored by the vast majority of software models, have strong impact on the classification performance of the network, and on its memory storage capacity. It has been demonstrated that the number of random uncorrelated patterns p which can be classified or stored in a network of neurons connected by bounded synapses grows only logarithmically with the number of synapses [9]. The number of patterns p grows quadratically with the number of states the synapse has to traverse to go from the lower bound to the upped bound (n). But this happens only in unrealistic scenarios where fine tuning of the network's parameters is allowed. In more realistic scenarios where there are inhomogeneities and variability (as is the case for biology and silicon) p is largely independent of n [9]. Therefore, an efficient strategy for implementing long lasting memories in VLSI networks of spiking neurons is to use a large number of bistable synapses (i.e. n = 2), and to modify their weights in a stochastic manner, with a small probability. This slows down the learning process, but has the positive effect of protecting previously stored memories from being overwritten. Using this strategy we can build large networks of spiking neurons with very compact learning circuits (e.g. that do not require local Analog-to-Digital Converters or floating gate cells for storing weight values). In addition, given the requirement of stochastic selection of synapses, we do not have to fight against technology to reduce the effect of inhomogeneities present in VLSI (e.g. induced by transistor mismatch), but could use it to our advantage. By construction, these types of devices operate in a massively parallel fashion and are fault-tolerant: even if a considerable fraction of the synaptic circuits is faulty due to fabrication problems, the overall functionality of the chip is not compromised. This can be a very favorable property in view of the potential problems of future scaled VLSI processes. The VLSI test chip used to carry out classification experiments implementing such strategy is shown in Fig. 1. The chip comprises 16 low-power integrate-and-fire (I&F) neurons [5] and 2048 dynamic synapses. It was fabricated using a standard 0.35µ m CMOS technology, and occupies an area of 6.1mm2 . We use an AER communication infrastructure that allows the chip to receive and transmit asynchronous events (spikes) off-chip to a workstation (for data logging and prototyping) and/or to other neuromorphic event-based devices [10]. An on-chip multiplexer can be used to reconfigure the neuron's internal dendritic tree connectivity. A single neuron can be connected to 128, 256, 512 or 1024 synapses. Depending on the multiplexer state the number of active neurons decrease from 16 to 2. In this work we configured the chip to use all 16 neurons with 128 synapses per neuron. The synapses are divided into different functional blocks: 4 are excitatory with fixed (externally adjustable) weights, 4 inhibitory and 120 excitatory with local learning circuits. 2


Soma

synapses

V UP

IUP
CC2 +

Vmth Vmem

S1

VUP Vmem

Ik2
CC1

Ik1 IB

bistable pre AER input w
DPI

VDN
CC3

Ik3 DPI I [Ca] I&F block Vspk

IEPSC
V DN

IDN Stop Learning

S2

(a)

(b)

Figure 2: (a) Plastic synapse circuits belonging to the neuron's dendritic tree. The synaptic weight node w is modified when there is a pre-synaptic input (i.e. when S1 and S2 are on) depending on the values of VU P and VDN . In parallel, the bistable circuit slowly drives the node w toward either of its two stable states depending on its amplitude. The DPI circuit produces an Excitatory PostSynaptic Current (IE PSC ) at each pre-synaptic spike, with an amplitude that depends on the synaptic weight w. (b) Soma with stop-learning module. It comprises a low-power I&F neuron block, a DPI integrator, a voltage comparator and a three current comparators(CC). Winner-take-all (WTA) circuits are used as current comparators that set the output to be either the bias current IB , or zero. The voltage comparator enables either the IU P or the IDN block, depending on the value of Vmem with respect to Vmt h . The voltages VU P and VDN are used to broadcast the values of IU P and IDN to the neuron's dendritic tree.

Every silicon neuron in the chip can be used as a classifier that separates the input patterns into two categories. During training, the patterns to be classified are presented to the pre-synaptic synapses, in parallel with a teacher signal that represents the desired response. The post-synaptic neuron responds with an activity that is proportional to its net input current, generated by the input pattern weighted by the learned synaptic efficacies, and by the teacher signal. If the neuron's mean activity is in accordance with the teacher signal (typically either very high or very low), then the output neuron produces the correct response. In this case the the synapses should not be updated. Otherwise, the synapses are updated and eventually make a transition to one of the two stable states with some probability. Such stochasticity, in addition to the 'stop-learning' mechanism which prevents the synapses from being modified when the output is correct, allows each neuron to classify a wide class of highly correlated, linearly separable patterns. Furthermore, by using more than one neuron per class, it is possible to classify also complex non-linearly separable patterns [4].

3

The VLSI learning circuits

The learning circuits are responsible for locally updating the synaptic weights with the spike-based learning rule proposed in [4]. Upon the arrival of a pre-synaptic spike (an address-event), the plastic synapse circuit updates its weight w according to the spike-driven learning rule. The synapse then produces an Excitatory Post-Synaptic Current (EPSC) with an amplitude proportional to its weight, and with an exponential time course that can be set to last from microseconds to several hundreds of milliseconds [11]. The EPSC currents of all synapses afferent to the target neuron are summed into the neuron's membrane capacitance, and eventually the I&F neuron's membrane potential exceeds a threshold and the circuit generates an output spike. As prescribed by the model of [4], the post-synaptic neuron's membrane potential, together with its mean firing rate are used to determine the weight change values w. These weight change values are expressed in the chip as subthreshold currents. Specifically, the signal that triggers positive weight updates is represented by an IU P current, and the signal that triggers weight decreases if represented by the IDN current. 3


3.2 3 2.8 2.6 2.4 0 3.2 3 2.8 2.6 0 0.4

Vmem

VCa

1 0 0 3

0.5

1

1.5

0.01

0.02

0.03

0.04

0.05

VDN

VDN

2.8 2.6 0 0.4 0.01 0.02 0.03 0.04 0.05

0.5

1

1.5
VUP

VUP

0.2 0 0 0.5 Time (s) 1 1.5

0.2 0 0 0.01 0.02 0.03 Time (s) 0.04 0.05

(a)

(b)

Figure 3: Post-synaptic circuit data. (a) State of the VU P and VDN voltages as a function of the calcium concentration voltage VCa . (b) State of the VU P and VDN voltages as function of the membrane potential Vmem . This data corresponds to a zoomed-version of the data shown in (a) for VCa  2.8V . The weight updates are performed locally at each synapse, in a pre-synaptic weight update module, while the w values are computed globally (per each neuron), in a post-synaptic weight control module. 3.1 Pre-synaptic weight-update module

This module, shown in Fig. 2(a), comprises four main blocks: an input AER interfacing circuit [12], a bistable weight refresh circuit, a weight update circuit and a log-domain current-mode integrator, dubbed the "diff-pair integrator" (DPI) circuit, and fully characterized in [11]. Upon the arrival of an input event (pre-synaptic spike), the asynchronous AER interfacing circuits produce output pulses that activate switches S1 and S2. Depending on the values of IU P and IDN , mirrored from the postsynaptic weight control module, the node w charges up, discharge toward ground, or does not get updated. The same input event activates the DPI circuit that produces an EPSC current (IE PSC ) with an amplitude that depends on the synaptic weight value w. In parallel, the bistable weight refresh circuit slowly drives w toward one of two stable states depending on whether it is higher or lower than a set threshold value. The two stable states are global analog parameters, set by external bias voltages. 3.2 Post-synaptic weight control module This module is responsible for generating the two global signals IU P and IDN , mirrored to all synapses belonging to the same dendritic tree. Post-synaptic spikes (Vs pk ), generated in the soma are integrated by an other instance of the DPI circuit to produce a current ICa proportional to the neuron's average spiking activity. This current is compared to three threshold values, Ik1 , Ik2 , and Ik3 of Fig. 2(b), using three current-mode winner-take-all circuits [13]. In parallel, the instantaneous value of the neuron's membrane potential Vmem is compared to the threshold Vmt h (see Fig. 2(b)). The values of IU P and IDN depend on the state of the neuron's membrane potential and its average frequency. Specifically, if Ik1 < ICa < Ik3 and Vmem > Vmt h , then IU P = IB . If Ik1 < ICa < Ik2 and Vmem < Vmt h , then IDN = IB . Otherwise both IU P , and IDN are null. To characterize these circuits we injected a step current in the neuron, produced a regular output mean firing rate, and measured the voltages VCa , VU P , and VDN (see Fig. 3(a)). VCa is the gate voltage of PMOS transistor producing ICa , while VDN , VU P are the gate voltages of a P and NMOS transistors mirroring IDN and IU P respectively (Fig. 2(a)). The neuron's spikes are integrated and the output current ICa increases with an exponential profile over time (VCa decreases accordingly over time, as shown in Fig. 3(a)). The steady-state asymptotic value depends on the average input frequency, as well as the circuit's bias parameters [11]. As ICa becomes larger than the first threshold Ik1 (VCa decreases below the corresponding threshold voltage) both VU P and VDN are activated. When ICa becomes larger than the second threshold Ik2 the VDN signal is deactivated, and finally 4


1.5 1 0.5 0 0 3 2.5 2 1.5 0 2 0 0

0.05

0.1

0.15

0.2

0.25 Vw

0.05

0.1

0.15

0.2

0.25

1.5 1 0.5 0 0 3 2.5 2 1.5 0 2 0 0

Vmem

Vmem

0.05

0.1

0.15

0.2

0.25

Vw

0.05

0.1

0.15

0.2

0.25

pre

0.05

0.1 0.15 Time(s)

0.2

0.25

pre

0.05

0.1 0.15 Time(s)

0.2

0.25

(a)

(b)

Figure 4: Stochastic synaptic LTP transition: in both sub-figures the non-plastic synapse is stimulated with Poisson distributed spikes at a rate of 250Hz, making the post-synaptic neuron fire at approximately 80Hz; and the plastic synapse is stimulated with Poisson distributed spike trains of 100Hz. (a) The updates in the synaptic weight did not produce any LTP transition during the 250ms stimulus presentation. (b) The updates in the synaptic weight produced an LTP transition that remains consolidated. as ICa becomes larger than the third threshold Ik3 , also the VU P signal is switched off. The small  300mV changes in VU P and VDN produce subthreshold currents (IU P and IDN ) that are mirrored to the synapses (Fig. 2(a)). In Fig. 3(b) the VDN and VU P signals are zoomed in along with the membrane potential of the post-synaptic neuron (Vmem ), for values of VCa  2.8V . Depending on the state of Vmem , the signals VU P and VDN are activated or inactivated. When not null, currents IU P and IDN are complementary in nature: only one of the two is equal to IB .

4

Stochastic plasticity

To characterize the stochastic nature of the weight update process we stimulated the neuron's plastic synapses with synthetic Poisson distributed spike trains. When an irregular Poisson distributed spike train is used as a pre-synaptic input, the synaptic weight voltage crosses the synapse bistability threshold in a stochastic manner, and the probability of crossing the threshold depends on the input's mean frequency. Therefore Long Term Potentiation (LTP) or Long Term Depression (LTD) occur stochastically even when the mean firing rates of the input and the output do not change. In Fig. 4 we show two instances of a learning experiment in which the mean input firing rate (bottom row) was 100Hz, and the mean output firing rate (top row) was 80Hz. Although these frequencies were the same for both experiments, LTP occurred only in one of the two cases (compare synaptic weight changes in middle row of both panels). In this experiment we set the efficacy of the "high" state of all plastic synapses to a relatively low value. In this way the neuron's mean output firing rate depends primarily on the teacher signal, irrespective of the states of plastic synapses. One essential feature of this learning rule is the non-monotonicity of both the LTP/LTD probabilities as a function of the post-synaptic firing frequency  post [4]. Such a non-monotonicity is essential to slow down and eventually stop-learning when  post is very high or very low (indicating that the learned synaptic weights are already correctly classifying the input pattern). In Fig. 5 we show experimental results where we measured the LTP and LTD transitions of 60 synapses over 20 training sessions: for the LTD case (top row) we initialized the synapses to a high state (white pixel) and plotted a black pixel if its final state was low, at the end of the training session. The transitions (white to black) are random in nature and occur with a probability that first increases and then decreases with  post . An analogous experiment was done for the LTP transitions (bottom row), but with complementary settings (the initial state was set to a low value). In Fig. 5(b) we plot the LTD (top row) and LTP (bottom row) probabilities measured for a single synapse. The shape of these curves can be modified by acting on the post-synaptic weight control module bias parameters such as Ik1-k3 , or IB . 5


5 Synapse number 1 20 40

20

100

180

320

500

700

900
1 p(LTD) p(LTP)

0.5

0

Synapse number

1 20 40

1

0.5

0 0

200

1

20

400 post(Hz)

600

(a)

(b)

Figure 5: (a) LTD and LTP transitions of 60 synapses measured across 20 trials, for different values of post-synaptic frequency  post (top label on each panel). Each black pixel represents a low synaptic state, and white pixel a high one. On x-axis of each panel we plot the trial number (1 to 20) and y-axis shows the state of the synapses at the end of each trial. In the top row we show the LTD transitions that occur after initializing all the synapses to high state. In the bottom row we show the LTP transition that occur after initializing the synapses to low state. The transitions are stochastic and the LTP/LTD probabilities peak at different frequencies before falling down at higher  post validating the stop-learning algorithm. No data was taken for the gray panels. (b) Transition probabilities measured for a single synapse as a function  post . The transition probabilities can be reduced by decreasing the value of IB . The probability peaks can also be modified by changing the biases that set Ik1-k3 . (Fig. 2(b))
+T

T-

Excitatory synapse, non-plastic Inhibitory synapse, non-plastic Excitatory synapse, plastic High input state (30Hz) Low input state (2Hz) Integrate and Fire neuron C+ C

Figure 6: A typical training scenario with 2 random binary spatial patterns. High and low inputs are encoded with generate Poisson spike trains with mean frequencies of 30Hz and 2Hz respectively. Binary patterns are assigned to the C+ or C- class arbitrarily. During training patterns belonging to the C+ class are combined with a T + (teacher) input spike train of with 250Hz mean firing rate. Similarly, patterns belonging to the C- class are combined with a T - spike train of 20Hz mean firing rate. New Poisson distributed spike trains are generated for each training iterations.

5

Classification of random spatial patterns

In order to evaluate the chip's classification ability, we used spatial binary patterns of activity, randomly generated (see Fig. 6). The neuron's plastic synapses were stimulated with Poisson spike trains of either high (30Hz) or low (2Hz) mean firing rates. The high/low binary state of the input was chosen randomly, and the number of synapses used was 60. Each 60-input binary pattern was then randomly assigned to either a C+ or a C- class. During training, spatial patterns belonging to the C+ class are presented to the neuron in conjunction with a T + teacher signal (i.e. a 250Hz Poisson spike train). Conversely patterns belonging to the C- class are combined with a T - teacher signal of 20Hz. The T + and T - spike trains are presented to the neuron's non-plastic synapses. Training sessions with C+ and C- patterns are interleaved in a random order, for 50 iterations. Each stimulus presentation lasted 500ms, with new Poisson distributions generated at each training session. 6


After training, the neuron is tested to see if it can correctly distinguish between patterns belonging to the two classes C+ and C- . The binary patterns used during training are presented to the neuron without the teacher signal, and the neuron's mean firing rate is measured. In Fig. 7(a) we plot the responses of two neurons labeled neuron-A and neuron-B. Neuron-A was trained to produce a high output firing rate in response to patterns belonging to class C+ , while neuron-B was trained to respond to patterns belonging to class C- . As shown, a single threshold (e.g. at 20Hz) is enough to classify the output in C+ (high frequency) and C- (low frequency) class.
80 p(post) p(post) 1 234 neuron-A 1 234 neuron-B 0.5

60 post(Hz)

40

0 0.5

20

0

0

0

50

100 post(Hz)

150

(a)

(b)

Figure 7: Classification results, after training on 4 patterns. (a) Mean output frequencies of neurons trained to recognize class C+ patterns (Neuron-A), and class C- patterns (Neuron-B). Patterns 1, 2 belong to class C+ , while patterns 3, 4 belong to class C- . (b) Output frequency probability distribution, for all C+ patterns (top) and C- patterns (bottom) computed over 20 independent experiments. Fig. 7(b) shows the probability distribution of post-synaptic frequencies (of neuron-A) over different classification experiments, each done with new sets of random spatial patterns. To quantify the chip's classification behavior statistically, we employed a Receiver Operating Characteristics (ROC) analysis [14]. Figure 8(a) shows the area under the ROC curve (AUC) plotted on y-axis for increasing number of patterns. An AUC magnitude of 1 represents 100% correct classification while 0.5 represents chance level. In Fig. 8(b) the storage capacity (p) ­expressed as the number of patterns with AUC larger than 0.75­ is plotted against the number f synapses used o N . The top and bottom traces show theoretical predictions from [4], with (p 2 N ) and without  (p N ) the stop learning condition, respectively. The scaling properties of the VLSI system with 20, 40 and 60 synapses lie within the theoretical boundaries.

6

Conclusions

We implemented in a neuromorphic VLSI device a recently proposed spike-driven synaptic plasticity, model that can classify complex patterns of spike trains [4]. We presented results from the VLSI chip that demonstrate the correct functionality of the spike-based learning circuits, and performed classification experiments of random uncorrelated binary patterns, that confirm the theoretical predictions. Additional experiments have demonstrated that the chip can be applied to the classification of correlated spatial patterns of mean firing rates and as well [15]. To our knowledge, the classification performance achieved with this chip has not yet been reported for any other silicon system. These results show that the device tested can perform real-time classification of sequences of spikes, and is therefore an ideal computational block for adaptive neuromorphic sensory-motor systems and brain-machine interfaces.

Acknowledgment
This work was supported by the Swiss National Science Foundation grant no. PP00A106556, the ETH grant no. TH02017404, and by the EU grants ALAVLSI (IST-2001-38099) and DAISY (FP62005-015803). 7


1
15

0.9
Storage capacity

AUC

0.8 0.7 0.6 0.5 2

10

5

4

6 8 # patterns

10

12

0

20

40 # input synapses

60

(a)

(b)

Figure 8: (a). Area under ROC curve (AUC) measured by performing 50 classification experiments. (b) Storage capacity (number of patterns with AUC value  0.75) as a function of the number of plastic synapses used. The solid line represents the data obtained from chip, while top and bottom traces represent the theoretical predictions with and without the stop learning condition.

References
¨ [1] R. Gutig and H. Sompolinsky. The tempotron: a neuron that learns spike timing­based decisions. Nature Neuroscience, 9:420­428, 2006. ¨ [2] R.A. Legenstein, C. Nager, and W. Maass. What can a neuron learn with spike-timing-dependent plasticity? Neural Computation, 17(11):2337­2382, 2005. [3] W. Senn and S. Fusi. Learning Only When Necessary: Better Memories of Correlated Patterns in Networks with Bounded Synapses. Neural Computation, 17(10):2106­2138, 2005. [4] J. Brader, W. Senn, and S. Fusi. Learning real world stimuli in a neural network with spike-driven synaptic dynamics. Neural Computation, 19:2881­2912, 2007. [5] G. Indiveri, E. Chicca, and R. Douglas. A VLSI array of low-power spiking neurons and bistable synapses with spike­timing dependent plasticity. IEEE Transactions on Neural Networks, 17(1):211­221, Jan 2006. ¨ [6] J. Arthur and K. Boahen. Learning in silicon: Timing is everything. In Y. Weiss, B. Scholkopf, and J. Platt, editors, Advances in Neural Information Processing Systems 18. MIT Press, Cambridge, MA, 2006. [7] D. Badoni, M. Giulioni, V. Dante, and P. Del Giudice. An aVLSI recurrent network of spiking neurons with reconfigurable and plastic synapses. In Proceedings of the IEEE International Symposium on Circuits and Systems, pages 1227­1230. IEEE, IEEE, May 2006. [8] G. Indiveri and S. Fusi. Spike-based learning in VLSI networks of integrate-and-fire neurons. In Proc. IEEE International Symposium on Circuits and Systems, ISCAS 2007, pages 3371­3374, 2007. [9] S. Fusi and L.F. Abbott. Limits on the memory storage capacity of bounded synapses. Nature Neuroscience, 2006. (In press). ¨ [10] E. Chicca, P. Lichtsteiner, T. Delbruck, G. Indiveri, and R.J. Douglas. Modeling orientation selectivity using a neuromorphic multi-chip system. In Proceedings of IEEE International Symposium on Circuits and Systems, pages 1235­1238. IEEE, 2006. [11] C. Bartolozzi and G. Indiveri. Synaptic dynamics in analog VLSI. Neural Computation, 19:2581­2603, Oct 2007. [12] K. A. Boahen. Point-to-point connectivity between neuromorphic chips using address-events. IEEE Transactions on Circuits and Systems II, 47(5):416­34, 2000. [13] J. Lazzaro, S. Ryckebusch, M.A. Mahowald, and C.A. Mead. Winner-take-all networks of O(n) complexity. In D.S. Touretzky, editor, Advances in neural information processing systems, volume 2, pages 703­711, San Mateo - CA, 1989. Morgan Kaufmann. [14] T. Fawcett. An introduction to ROC analysis. Pattern Recognition Letters, (26):861­874, 2006. [15] S. Mitra, G. Indiveri, and S. Fusi. Robust classification of correlated patterns with a neuromorphic VLSI network of spiking neurons. In IEEE Proceedings on Biomedical Circuits and Systems (BioCAS08), 2008. (In press).

8