Abstract
We have extended a previous spiking neural network model of prefrontal cortex with fast Hebbian plasticity to include also interactions between short-term and long-term cortical memory stores. We investigated how prefrontal cortex could bind, maintain and update multi-modal long-term memory representations by simulating three cortical patches in macaque brain, corresponding to networks in prefrontal cortex together with visual and auditory temporal cortical areas.
Our simulation results demonstrate how simultaneous, brief multi-modal memory cues could build a temporary joint memory representation linked via prefrontal cortex. The latter can then activate spontaneously and thereby reactivate the associated long-term representations. Cueing one long-term memory item rapidly pattern-completes the associated un-cued item via prefrontal cortex. The short-term memory network flexibly updates as new stimuli arrive thereby gradually over-writing older representations. In a wider context, this working memory model suggests a novel explanation for the ‘neural binding problem’, a long-standing and fundamental issue in cognitive neuroscience.
Introduction
By working memory (WM) we typically understand a flexible but volatile kind of memory capable of holding a small number of items over shorter time spans. It brings task relevant information in long-term memory (LTM) on-line and enables acting beyond the immediate here and now. WM is a key component in cognition and is often affected early on in neurological and psychiatric conditions, e.g. Alzheimer’s disease and schizophrenia1. Prefrontal cortex (PFC) has repeatedly been implicated as a key neural substrate for WM in humans and non-human primates2,3.
For thirty years the prevalent theory of WM has been that of persistent activity in PFC recurrent networks, proposed by Goldman-Rakic and colleagues4,5 and supported by computational models6,7. Although there is solid support for delay activity in PFC during WM, recent experimental observations, especially of the intermittent and bursty nature of this activity8,9 and “activity silent working memory”10 have cast doubt on persistent activity as the main neural mechanism underlying WM.
Alternative theories and models of WM have favored Hebbian synaptic plasticity as a key underlying neural mechanism. Early computational models employing non-spiking neural units demonstrated a proper WM function11–15, but required rapid induction and expression of plasticity, judged incompatible with contemporary experimental observations on long-term potentiation (LTP). In 2008, Mongillo et al.16 proposed a “synaptic working memory theory” and demonstrated a spiking network model of cortex that displayed WM maintenance in the form of self-sustained bursty activity. This model relied on non-Hebbian forms of synaptic plasticity and met the timing constraints. However, its synaptic weight changes are independent of postsynaptic activity and can thus only facilitate already encoded memory items, which precludes encoding of novel memories.
Building on an earlier abstract model18, we recently modeled a human word-list learning task using a spiking short-term memory (STM) model of PFC employing fast Hebbian synaptic plasticity19. We demonstrated our model’s capability to encode novel items, to quantitatively reproduce key characteristics of human cued and non-cued recall, and to reproduce several important network dynamic features of memory replay and recall. To function in this context, synaptic plasticity needs to operate on a time-scale of a few hundred milliseconds and recent experiments have demonstrated the existence of such fast forms of Hebbian synaptic plasticity, e.g. short-term potentiation (STP) 20–22, which lends credibility to this type of mechanism.
The new WM theory presented here builds on this previous work but extends it in a new direction by integrating STM in PFC with multi-modal LTM in parieto-temporal cortical areas. Fast Hebbian synaptic plasticity is a critical player also in this dynamic interaction. We implement this theory in a spiking neural network model and demonstrate how WM emerges as a tight co-operation between STM and LTM. This allows for one-shot encoding, maintenance and recall of memory representations as well as flexible updating of which ones are currently facilitated or on-line. Interestingly, this novel theory and model also unifies WM with the classical phenomenon of “neural binding” and suggests an explanation for fundamental perceptual and cognitive processes like feature binding and role binding.
Results
Our model is particularly concerned with the study of multi-modal memory binding and interactions across the cortical hierarchy. To this end, we simulate three cortical patches with significant biophysical detail: an STM and two LTM networks (LTMa, LTMb), representing PFC and temporal visual and auditory cortical areas respectively (Figure 1). The computational network model used here represents a detailed modular cortical microcircuit architecture in line with previous models23,24. In particular, the current model is built upon a recent STM model19. The sub-sampled associative cortical layer 2/3 network of that model was sub-divided into layers 2, 3A, and 3B and extended with an input layer 4 and corticocortical connectivity to LTM stores in temporal cortical regions. This large, composite model synthesizes many different kinds of cortical data and produces complex output dynamics. We specifically focus on the dynamics of memory specific subpopulations in the interaction of STM and LTM stores.
We introduce the operation of the corticocortical model in several steps. First, we take a brief look at background activity and active memory states in isolated cortical networks of this kind to familiarize the reader with some of its dynamical properties. Second, we describe the effect of memory activation on STM with and without plasticity. Third, we add the plastic backprojections from STM to LTM and monitor the encoding and maintenance of several memories in the resulting STM-LTM loop. We track the evolution of acquired cell assemblies with shared pattern-selectivity in STM and show their important role in WM maintenance (a.k.a. delay activity). We then demonstrate that the emerging WM network system stays plastic and capable of updating its maintained set of memories. Finally, we simulate multi-modal memory binding and analyze its dynamical correlates. We explore temporal characteristics of network activations and cross-cortical delays during WM encoding, maintenance, and cue-driven associative recall of memory pairs associated in this manner.
Background activity and Activated memory
In the untrained network, fluctuations in membrane voltages manifest a ground state with low-rate, irregular, asynchronous spiking that transitions into a global oscillation in the alpha/beta range at medium input levels (Figure 2-A). This is largely an effect of fast feedback inhibition from local basket cells (Supplementary Figure 1), high connection density within MCs, and low latency local spike transmission.25 If the network has been trained with structured input so as to encode memory (i.e. attractor states), a specific cue (Online Methods) can trigger memory item reactivations accompanied by fast oscillations modulated by an underlying slow oscillation in the theta range 26,27 (Figure 2-B). The spiking activity of memory activations (a.k.a. attractors) is short-lived due to neural adaptation and synaptic depression. When unspecific background excitation is very strong, this can result in a random walk across stored memories.
LTM-to-STM Forward Dynamics
We now consider cued activation of several memories embedded in LTM. Each HC in LTM features a selectively coding MC for any given memory pattern that activate synchronously in theta-like cycles containing several fast oscillation bursts (Figure 2-B). Five different LTM memory patterns are triggered by brief cues, accompanied by an upregulation of STM plasticity, see Figure 2-D (bottom). To indicate the spatio-temporal structure of evoked activations in STM, we also show a simultaneous subsampled STM spike raster (Figure 2-D top). STM activations are sparse (ca 5%), yet nearby cells (in the same MC) often fire together (even during background activity, Figure 2-A). The distributed, but patchy character of the STM response to memory activations (Figure 2-D top) is shaped by branching forward-projections from LTM layer 3B cells, which tend to activate close-by cells. STM input layer 4 receives half of these corticocortical connections and features very high fine-scale specificity in its projections to layer 2/3 pyramidal neurons, which furthers recruitment of local clusters with shared selectivity. STM cells initially fire less than those in LTM because the latter received a brief, but strong activation cue and feature strong recurrent weights between cells in embedded memory patterns. STM spikes in Figure 2-D are colored according to the cells dominant memory pattern selectivity, which reveals that STM activations are mostly non-overlapping as well. Although MC activity is not exclusive to any given input pattern (unlike the strictly orthogonal LTM patterns), nearby cells often have similar pattern selectivity. This is not only an effect of competition via basket cell feedback inhibition, but also a result of short-term dynamics, such as neural adaptation and synaptic depression. Neurons that were recently activated by a strong, bursting input from LTM are refractory and thus less prone to spike again for some time thereafter (τrec and , Supplementary Table 1), further reducing the likelihood of activating overlapping STM activation patterns. Figure 2-C shows a peri-stimulus population firing rate of both networks (mean across 100 trials with five triggered memories each). There is a bottom-up response delay between stimulus onset at t=0 and LTM activation, as well as a substantial forward delay (which we will scrutinize in more detail later on). Oscillatory activity in STM is lower than LTM mostly because the untrained STM lacks strong recurrent connections, is thus less excitable than LTM in our model, and therefore does not trigger its basket cells as quickly (the main drivers of fast oscillations in our model). STM oscillations and burst amplitude speed up within a few seconds as new cell assemblies become stronger (e.g. Figure 3-A and Supplementary Figure 2). As seen in Figure 2-B, bursts of co-activated MCs in LTM can become asynchronous during activation. Dispersed forward axonal conduction delays further decorrelate this gamma-like input to STM. Activating strong plasticity in STM (κ = κp, Online Methods and Supplementary Table 1) has a noticeable effect on the amplitude of stimulus-locked oscillatory STM activity after as little as 100 ms (cf. Figure 2-C, STM).
Multi-item Working Memory
In Figure 2-D, we showed emerging pattern-specific subpopulations in STM from feedforward input. Modulated STM plasticity allows for the quick formation of rather weak STM cell assemblies from one-shot learning. When we include plastic STM backprojections, these assemblies can become an index of sorts for specific memories. Their recruitment is temporary, but they can act as top-down control signals for memory maintenance and retrieval, as we will see now. By enabling backprojections, we can track multiple activated memories in the closed STM-LTM loop. In Figure 3-A, we show network activity following targeted activation of five LTM memories (Spike raster in Supplementary Figure 2). Under an increased unspecific noise-drive (, Supplementary Table 2), STM cell assemblies, formed during the brief plasticity-modulated stimulus phase (cf. Figure 2D) may activate spontaneously. These brief bursts of activity are initially weak and different from the theta-like cycles of repeated fast bursting seen in LTM attractor activity.
STM recurrent connections remain plastic (κ = 1) throughout the simulation, so each reactivation event further strengthens memory-specific cell assemblies in STM. As a result, there is a noticeable ramp-up in the strength of STM pattern-specific activity over the course of the delay period (cf. increasing burst length and amplitude in Figure 3-A, or Supplementary Figures 3, 5). STM backprojections are also plastic and thus acquire memory specificity from STM-LTM co-activations, especially during the initial stimulation phase. Given enough STM cell assembly firing, their sparse but potentiated backprojections can trigger associated memories. Weakly active assemblies may fail to do so. In the example of Figure 3-A, we can see a few early STM reactivations that are not accompanied (or quickly followed) by a corresponding LTM pattern activation (of the same color) in the first two seconds after the plasticity-modulated stimulation phase. When LTM is triggered, there is a noticeable feedback delay (Figure 3-C), which we will scrutinize later on.
Cortical feedforward and feedback pathways between LTM and STM form a loop, so each LTM activation will again feed into STM, typically causing a second peak of activation in STM 40 ms after the first (Figure 3-C). The forward delay from LTM to STM, we originally observed in the stimulus-driven input phase (Figure 2-C) is still evident here in this delayed secondary increase of the STM activation following LTM onset, which also extends/sustains the STM activation and helps stabilize memory-specific STM cell assemblies and their specificity. This effect may be called auto-consolidation and it is an emergent feature of the plastic STM-LTM loop in this model. It happens on a timescale governed by the unmodulated plasticity time constant (κ = κnormal, τp = 5 ѕ, Supplementary Table 1). After a few seconds, the network has effectively stabilized and typically maintains a small set of 3-4 activated long-term memories. The closed STM-LTM loop thus constitutes a functional multi-item WM.
A crucial feature of any WM system is its flexibility, and Figure 3-B highlights an example of rapid updating. The maintained set of activated memories can be weakened by stimulating yet another set of input memories. Generally speaking, earlier items are reliably displaced from active maintenance in our model if activation of the new items is accompanied by the same transient elevation of plasticity (κp/κnormal, Supplementary Table 1) used during the original encoding of the first five memories (Corresponding spike rasters and population firing rates are shown in Supplementary Figures 3, and 4).
In line with earlier results19, cued activation can usually still retrieve previously maintained items. The rate of decay for memories outside the maintained set depends critically on the amount of noise in the system, which erodes the learned associations between STM and LTM neurons as well as STM cell assemblies. We note that such activity-dependent memory decay is substantially different from time-dependent decay, as in Mi et al.29.
Multi-modal, Multi-item Working Memory
Next, we explore the ability of the closed STM-LTM loop system to flexibly bind co-active pairs of long-term memories from different modalities (LTMa and LTMb respectively). As both LTM activations trigger cells in STM via feedforward projections, a unique joint STM cell assembly with shared pattern-selectivity is created. Forward-activations include excitation and inhibition and combine non-linearly with each other (see Online Methods) and with prior STM content. Figure 4 illustrates how this new index then supports WM operations, including delay maintenance through STM-paced discrete co-activation events and cue-driven associative pair completion. The three columns of Figure 4 illustrate three fundamental modes of the closed loop system: stimulus driven binding, WM maintenance, and associative recall. The top three rows show sampled activity of a single trial (see also Supplementary Figures 5,6), whereas the bottom row shows multi-trial averages.
During stimulus-driven binding, we co-activate memories from both LTM’s. As before, when we used only one LTM network, this involves brief 50 ms cues that trigger activation of corresponding memory patterns. The multi-trial (and multi-pattern) average of peri-stimulus activations reveals a 45 ± 7.3 ms attractor activation delay, followed by a 43 ± 7.8 ms feedforward delay (about half of which is explained by the spatial distance between LTM and STM) from the onset of the two LTM activations to the onset of the input-specific STM response (Figure 5 top-left and bottom-left).
During WM maintenance, a 10 s delay period, paired LTM memories reactivate together. Onset of these paired activations is a lot more variable than during cued activation, mostly because the driving STM cell assemblies are of variable size and strength, which has an impact on the wider feedback delay distribution, with mean 41.5 ± 15.3 ms.
Following the maintenance period, we test the memory system’s ability for associative recall. To this end, we cue LTMa, again using a targeted 50 ms cue for each memory, and track the systems response across the STM-LTM loop. We compute multi-trial averages of peri-stimulus activations during recall testing (Figure 4 bottom-right). Following cued activation of LTMa, STM responds with the related joint cell assembly as the input is strongly correlated to learned inputs from the simultaneous activation with LTMb earlier on. Similar to the mnemonic function of an index, the completed STM pattern then triggers the associated memory in LTMb through its backprojections. STM activation now extends far beyond the transient activity of LTMa because both the STM recurrent connectivity and then the STM-LTMb backprojection re-excite it. Temporal overlap between associated LTMa and LTMb memory activations peaks around 125 ms after the initial stimulus to LTMa.
We collect distributions of feedforward and feedback delays during associative recall (Figure 5). To facilitate a more immediate comparison with biological data we also computed the Bottom-Up and Top-Down response latency of the model in analogy to Tomita et al.30. Their study explicitly tested widely held beliefs about the executive control of PFC over ITC in memory retrieval. To this end, they identified and recorded neurons in ITC of monkeys trained to memorize several visual stimulus-stimulus associations. They employed a posterior-split brain paradigm to cleanly disassociate the timing of the bottom-up (contralateral stimuli) and top-down response (ipsilateral stimuli) in 43 neurons significantly stimulus-selective in both conditions and observed that the latency of the top-down response (178 ms) was longer than that of the bottom-up response (73 ms).
Our simulation is analogous to this experimental setup with respect to some key features, such as the spatial extent of memory areas (about 289 mm²) and inter-area distances (40 mm cortical distance between PFC and ITC). These measures heavily influence the resulting connection delays and time needed for information integration. In analogy to the posterior-split brain experiment our model’s LTMa and LTMb are unconnected. The display of a cue in one hemi-field in the experiment corresponds to the LTMa-sided stimulation of an associated memory pair in the model. This arrangement forces any LTM interaction through STM (representing PFC), and allows us to treat the cued LTMa memory activation as a Bottom-up response, whereas the much later activation of the associated LTMb representation is related to the Top-down response in the experimental study. Figure 5 shows the distribution of these latencies in our simulations, where we also marked the mean latencies measured by Tomita et al. The mean of our bottom-up delay (72.9 ms) matches the experimental data (73 ms), whereas the mean of the broader top-down latency distribution (155.2 ms) is a bit lower than in the monkey study (178 ms). Of these 155.2 ms, only 48 ms are explained by the spatial distance between networks, as verified by a fully functional alternative model with 0 mm distance between networks.
Discussion
We have presented a novel theory for WM and the design and evaluation of a spiking neural network WM model featuring STM (PFC) and LTM components (parieto-temporal cortex) as well as fast Hebbian synaptic plasticity. We demonstrated how single memory items can be encoded by one-shot learning and how PFC rapidly enhances synaptic connections intrinsically as well as those targeting LTM representations, which are thereby “brought on-line”. The PFC cell assembly serves as a temporary index into LTM in a similar manner as suggested by the “Hippocampal Indexing Theory” for the relation between hippocampus and neocortex in episodic memory and memory consolidation17,31. When a pair of LTM items are bound together by being simultaneously indexed in this manner, activating one will also activate the other indirectly via PFC with a short latency. We further demonstrated that this kind of WM is readily updated such that as new items are encoded, old ones are gradually over-written.
Recall dynamics in the model presented is in most respects identical to our previous cortical associative memory models32. Any activated memory item, whether randomly or specifically triggered, is subject to known and previously well characterized associative memory dynamics, such as pattern completion, rivalry, bursty reactivation dynamics, oscillations in different frequency bands, etc.28–30. Moreover, sequential learning and recall could readily be incorporated in the model33. This could for example support encoding of sequences of items in WM rather than unrelated single ones, resulting in a reactivation dynamic reminiscent of the “phonological loop”34.
The Case for Hebbian Plasticity
The underlying mechanism of our model is fast Hebbian plasticity, not only in the intrinsic PFC connectivity, but also in the projections from PFC to LTM stores. The former has some experimental support20,21,35,36 whereas the latter remains a prediction of the model. Dopamine D1 receptor (D1R) activation by dopamine (DA) is strongly implicated in reward learning and synaptic plasticity regulation in the basal ganglia37. In analogy we propose that D1R activation is also critically involved in the synaptic plasticity intrinsic to PFC and in projections to LTM stores, which would also explain prominent WM effects of PFC DA level manipulation38,39. In our model, κ represents the level of DA-D1R activation, which in turn regulates its synaptic plasticity. We typically increase kappa 4-8 fold temporarily in conjunction with LTM stimulation and WM encoding, in a form of attentional gating. Larger modulation limits WM capacity to 1-2 items, while less modulation diminishes the strength of cell assemblies beyond what is necessary for reactivation and LTM maintenance.
When the synaptic plasticity WM hypothesis was first presented and evaluated, it was based on synaptic facilitation16,26. However, such non-Hebbian plasticity is only capable of less specific forms of memory. Activating a cell assembly, comprising a subset of neurons in an untrained STM network, would merely facilitate all outgoing synapses from active neurons. Likewise, an enhanced elevated resting potential resulting from intrinsic plasticity would make the targeted neurons more excitable. In either case, there would be no coordination of activity specifically within the stimulated cell assembly. Thus, if superimposed on an existing LTM, such forms of plasticity may well contribute to WM, but are by themselves not capable of supporting encoding of novel memory items or the multi-modal binding of already existing ones. In our previous paper19 we showed that fast Hebbian plasticity in the form of STP20 allows such one-shot encoding of new STM items. In the current extended model, by also assuming the same kind of plasticity in backprojections from PFC to parieto-temporal LTM stores, PFC can also bind and bring on-line previously unassociated LTM items across multiple modalities.
Our implementation of a fast Hebbian plasticity reproduces a remarkable aspect of STP: it decays in an activity-dependent manner35,36. Its decay is not noticeably time-dependent, and silence preserves synaptic information. The typically detrimental effects of distractors on performance in virtually all kinds of WM tasks suggest an activity-dependent update, as does the duration of “activity-silent WM” in recent experiments10. Although we used the BCPNN learning rule to reproduce these effects, we expect that other Hebbian learning rules allowing for neuromodulated fast synaptic plasticity could give comparable results.
Experimental support and Testable predictions
Our model is built from available relevant microscopic data on neural and synaptic components as well as modular structure and connectivity of the targeted cortical areas in macaque monkey. When challenged with specific stimulus items, the network so designed generates a well-organized macroscopic dynamic working memory function, which can be interpreted in terms of manifest behavior and validated against cognitive experiments and data.
Unfortunately, the detailed neural processes and dynamics of our new model are not easily accessible experimentally and it is therefore quite hard to find direct and quantitative results to validate it. Yet, in analyzing our resulting bottom-up and top-down delays, we drew an analogy to a split-brain experiment30 because of its clean experimental design (even controlling for subcortical pathways) and found remarkably similar temporal dynamics in our highly subsampled cortical model. The timing of inter-area signals also constitutes a testable prediction for multi-modal memory experiments. Furthermore, reviews of intracranial recordings conclude that theta band oscillations play an important role in long-range communication during successful retrieval43. With respect to theta band oscillations in our model, STM leads the rest of cortex during maintenance, engages bi-directionally during recall (due to the STM-LTM loop), and lags during stimulus-driven encoding and LTM activation, reflecting experimental observations44. These effects can be explained by our model architecture, which imposes delays due to the spatial extent of networks and their distances from each other. Fast oscillations in the gamma band, while often theta-nested, are strongly linked to local processing and activated memory items in our model, also matching experimental findings43. Local frequency coupling is abundant with significant phase-amplitude coupling (e.g. Figure 2B), and was well characterized in related models27.
The most critical requirement and thus prediction of our theory and model is the presence of fast Hebbian plasticity in the PFC back-projections to parieto-temporal memory areas. Without such plasticity, our model cannot explain STM-LTM binding. This is likely to co-exist with neuromodulatory control over that plasticity, likely with DA and D1R activation involvement. Since the STP decays with activity a high noise level could be an issue since it could shorten WM duration (see The Case for Hebbian Plasticity). The evaluation of this requirement is hampered by little experimental evidence and a general lack of experimental characterization of the synaptic plasticity in long-range corticocortical projections.
Our model also makes specific predictions about corticocortical long-range connectivity. For example, as few as six active synapses (see Online Methods) onto each coding pyramidal neuron is sufficient to transfer specific memory identities across the cortical hierarchy and to support maintenance and recall.
Finally, our model suggests the occurrence of a double peak of frontal activation in executive control of LTM memories (see STM activity during WM Maintenance in Figure 4). The first one originates from the top-down control signal itself, and the second one is a result of corticocortical reentry and a successful activation of an associated item in LTM. As such, the second peak should also be correlated to successful memory maintenance or associative recall.
The Binding Problem
The classical so called binding problem refers to the brain’s ability to temporarily connect features or items previously not connected by earlier experience and learning45. The fast Hebbian plasticity proposed here as a mechanism for WM offers without any modification an additional plausible neural mechanism that solves the binding problem. For instance, visual feature binding refers to the instant association of for example shape and color of multiple objects without mixing them up during recall (Figure 6-A). Suggested neural mechanisms include temporal binding by synchronization46 or feature integration47. As an alternative solution, the theory presented here suggests that the joint index to LTM areas formed in PFC/STM during presentation of respective stimulus pairs, serves to bind shape and color in a specific manner thus avoiding any confusion. Filling roles with instances, as in “Charlie is my parrot”, is a second form of variable-value binding (Figure 6-B). This and other forms of neural binding underlie even more complex functions in human cognition including logical reasoning48. Here as well, the joint STM index provides effective binding and its flexible updating prevents confusion in a case when the name of the boss is also Charlie. That PFC may be involved in the binding process is supported by the observation that temporary PFC inactivation in humans affects feature binding49, and that successful feature binding requires attention50, i.e. that the attentional gate is opened allowing for enhanced plasticity via dopaminergic innervation. Gamma band oscillations, frequently implicated when binding is observed, are also a prominent output of our model51.
Conclusions
Our simulations have demonstrated the versatile WM properties of a spiking neural network model composed of integrated STM and LTM components and we have partly connected them to existing experimental data. Although our model is still quite abstract, it nevertheless offers a biologically plausible mechanistic understanding of WM and of neural binding, which connects microscopic processes with macroscopic observations and function in a way that only computational models can do. While we applied this model to macaque, it is quite generic and we expect model findings to apply also to mammals including humans, commensurate with changes in key parameters (cortical distances, axonal conductance speeds, etc.).
WM dysfunction has an outsized impact on mental health, intelligence, and quality of life. Progress in mechanistic understanding of function and dysfunction is therefore very important for society. Our hope is that our new computational model, which reproduces several important aspects of human WM, will trigger many more targeted experiments aimed to evaluate and improve it. More adequate models and clarity can help position future research and development appropriately even in the clinical and pharmaceutical realm.
Conflict of Interest
Nothing to declare
Online Methods
Neuron Model
We use an integrate-and-fire point neuron model with spike-frequency adaptation52 which was modified41 for compatibility with a custom-made BCPNN synapse model in NEST through the addition of the intrinsic excitability current . The model was simplified by excluding the subthreshold adaptation dynamics. Membrane potential Vm and adaptation current are described by the following equations:
The membrane voltage changes through incoming currents over the membrane capacitance Cm. A leak reversal potential EL; drives a leak current through the conductance gL and an upstroke slope factor ΔT determines the sharpness of the spike threshold Vt. Spikes are followed by a reset of membrane potential to Vr. Each spike increments the adaptation current by b, which decays with time constant . Simulated basket cells feature neither the intrinsic excitability current nor this spike-triggered adaptation.
Besides external input Iext (Stimulation Protocol) neurons receive a number of different synaptic currents from its presynaptic neurons in the network (AMPA, NMDA and GABA), which are summed at the membrane accordingly:
Synapse Model
Excitatory AMPA and NMDA synapses have a reversal potential EAMPA = ENMDA, while inhibitory synapses drive the membrane potential toward EGABA. In addition to BCPNN learning (next Section), plastic synapses are also subject to synaptic depression (vesicle depletion) according to the Tsodyks-Markram formalism53:
The fraction of synaptic resources available at each synapse is depleted by a synaptic utilization factor (U) with each spike transmission and recovers with τrec back towards its maximum value of 1. Every presynaptic input spike (at with transmission delay tij) thus evokes a transient synaptic current through a change in synaptic conductance that follows an exponential decay with time constants τsyn depending on the synapse type (τAMPA ≪ τNMDA).
H(·) denotes the Heaviside step function, and is the peak amplitude of the conductance transient, learned by the following Spike-based BCPNN Learning Rule.
Spike-based BCPNN Learning Rule
Plastic AMPA and NMDA synapses are modeled to mimic short-term potentiation (STP)20 with a spike-based version of the Bayesian Confidence Propagation Neural Network (BCPNN) learning rule41,54. For a full derivation from Bayes rule, deeper biological motivation, and proof of concept, see Tully et al. (2014) and the earlier STM model implementation19.
Briefly, the BCPNN learning rule makes use of biophysically plausible local traces to estimate normalized pre-and post-synaptic firing rates, as well as co-activation, which can be combined to implement Bayesian inference because connection strengths and MC activations have a statistical interpretation13,17,41. Crucial parameters include the synaptic activation trace Z, which is computed from spike trains via pre-and post-synaptic time constants , which are the same here but differ between AMPA and NMDA synapses:
The larger NMDA time constant reflects the slower closing dynamics of NMDA-receptor gated channels. All excitatory connections are drawn as AMPA and NMDA pairs, such that they feature both components. Further filtering of the Z traces leads to rapidly expressing memory traces (referred to as P-traces) that estimate activation and coactivation:
These traces constitute memory itself and decay in a palimpsest fashion. STP decay is known to take place on timescales that are highly variable and activity dependent36; see Discussion – The case for Hebbian plasticity.
We make use of the learning rule parameter κ (Equation 7), which may reflect the action of endogenous neuromodulators, e.g. dopamine acting on D1 receptors, that signal relevance and thus modulate learning efficacy. It can be dynamically modulated to switch off learning to fixate the network, or temporarily increase plasticity (κp, κnormal, Supplementary Table 1). In particular, we trigger a transient increase of plasticity concurrent with external stimulation.
Tully et al. 41 show that Bayesian inference can be recast and implemented in a network using the spike-based BCPNN learning rule. Prior activation levels are realized as an intrinsic excitability of each postsynaptic neuron, which is derived from the post-synaptic firing rate estimate pj and implemented in the NEST neural simulator55 as an individual neural current with scaling constant βgain
is thus an activity-dependent intrinsic membrane current to the neurons, similar to the A-type K+ channel 56 or TRP channel 57. Synaptic weights are modeled as peak amplitudes of the conductance transient (Equation 5) and determined from the logarithmic BCPNN weight, as derived from the P-traces with a synaptic scaling constant .
In our model, AMPA and NMDA synapses make use of and respectively. The logarithm in Equations 8, 9 is motivated by the Bayesian underpinnings of the learning rule, and means that synaptic weights multiplex both the learning of excitatory and di-synaptic inhibitory interaction. The positive weight component is here interpreted as the conductance of a monosynaptic excitatory pyramidal to pyramidal synapse (Supplementary Figure 1, plastic connection to the co-activated MC), while the negative component (Supplementary Figure 1, plastic connection to the competing MC) is interpreted as di-synaptic via a dendritic targeting and vertically projecting inhibitory interneuron like a double bouquet and/or bipolar cell58–61. Accordingly, BCPNN connections with a negative weight use a GABAergic reversal potential instead, as in previously published models19,23,41. Model networks with negative synaptic weights have been shown to be functionally equivalent to ones with both excitatory and inhibitory neurons with only positive weights62.
Code for the NEST implementation of the BCPNN synapse is openly available (see Simulation Environment).
Axonal Conduction Delays
We compute axonal delays tij between presynaptic neuron i and postsynaptic neuron j, based on a constant conduction velocity V and the Euclidean distance between respective columns. Conduction delays were randomly drawn from a normal distribution with mean according to the connection distance divided by conduction speed and with a relative standard deviation of 15% of the mean in order to account for individual arborization differences. Further, we add a minimal conduction delay of 1.5 ms to reflect not directly modeled delays, such as diffusion of transmitter over the synaptic cleft, dendritic branching, thickness of the cortical sheet, and the spatial extent of columns:
STM Network Architecture
We simulate on a grid with spatial extent of 17×17 mm. This spatially distributed network of columns has sizable conduction delays due to the distance between columns and can be interpreted as a spatially distributed subsampling of columns from the extent of dorsolateral PFC (such as BA 46 and 9/46, which also have a combined spatial extent of about 289 mm² in macaque).
Each of the non-overlapping HCs has a diameter of about 640 μm, comparable to estimates of cortical column size63, contains 24 basket cells, and its pyramidal cell population has been divided into twelve functional columns (MC). This constitutes another sub-sampling from the roughly 100 MC per HC when mapping the model to biological cortex. We simulate 20 pyramidal neurons per MC to represent roughly the layer 2 population of an MC, 5 cells for layer 3A, 5 cells for layer 3B, and another 30 pyramidal cells for layer 4, as macaque BA 46 and 9/46 have a well-developed granular layer64. The STM model thus contains about 18.000 simulated pyramidal cells in four layers (although layers 2, 3A, and 3B are often treated as one layer 2/3).
STM Network Connectivity
The most relevant connectivity parameters are found in Supplementary Tables 1–3. Pyramidal cells project laterally to basket cells within their own HC via AMPA-mediated excitatory projections with a connection probability of pp-B, i.e. connections are randomly drawn without duplicates until the target fraction of all possible pre-post connections exist. In turn, they receive GABAergic feedback inhibition from basket cells (pB-p) that connect via static inhibitory synapses rather than plastic BCPNN synapses. This strong loop implements a competitive soft-WTA subnetwork within each HC 65. Local basket cells fire in rapid bursts, and induce alpha/beta oscillations in the absence of attractor activity and gamma, when attractors are present and active.
Pyramidal cells in layer 2/3 form connections both within and across HCs at connection probability pL23e-L23e. These projections are implemented with plastic synapses and contain parallel AMPA and NMDA components, as explained in subsection Spike-based BCPNN Learning Rule. Connections across columns and areas may feature sizable conduction delays due to the implied spatial distance between them (Online Methods and Supplementary Table 1)
Pyramidal cells in layer 4 project to pyramidal cells of layer 2/3, targeting 25% of cells within their respective MC only. Experimental characterization of excitatory connections from layer 4 to layer 2/3 pyramidal cells have confirmed similarly high fine-scale specificity in rodent cortex66 and in-turn, full-scale cortical simulation models without functional columns have found it necessary to specifically strengthen these connections to achieve defensible firing rates67.
In summary, the STM model thus features a total of 16.2 million plastic AMPA-and NMDA-mediated connections between 18.000 simulated pyramidal cells in STM, as well as 67.500 static connections from 9.000 layer 4 pyramidals to layer 2/3 targets within their respective MC, and 604.800 static connections to and from 600 simulated basket cells.
LTM network
We simulate two structurally identical LTM networks, referred to as LTMa, and LTMb. LTM networks may be interpreted as a spatially distributed subsampling of columns from areas of the temporal cortex commonly associated with auditory and visual LTM. For example Inferior Temporal Cortex (ITC) is often referred to as the storehouse of visual LTM68. Two such LTM areas are indicated in Figure 1.
We simulate in each area and nine MC per HC (further details in Supplementary Tables 1–3). Both LTM networks are structurally very similar to the previously described STM, yet they do not feature plasticity beyond short-term dynamics in the form of synaptic depression. Unlike STM, LTM areas also do not feature an input layer 4, but are instead stimulated directly to cue the activation of previously learned long-term memories (Stimulation Protocol). Various previous models with identical architecture have demonstrated how attractors can be learned via plastic BCPNN synapses18,19,23,41. We load each LTM network with nine orthogonal attractors (ten in the example of Figure 3-B, which features two sets of five memories each). Each memory pattern consists of 16 active MCs, distributed across the 16 HCs of the network. We load-in BCPNN weights from a previously trained network (Supplementary Table 2), but thereafter set κ = 0 to deactivate plasticity of recurrent connections in LTM stores.
In summary, the two LTM models thus feature a total of 7.46 million connections between 8.640 pyramidal cells, as well as 13.608 static connections to and from 576 basket cells.
Corticocortical Connectivity
Our model implements supragranular feedforward and feedback pathways, as inspired by recent characterizations of such pathways by Markov et al.69 between cortical areas that are at a medium distance in the cortical hierarchy. The approximate cortical distance between Inferior Temporal Cortex (ITC) and dlPFC in macaque is about 40 mm and with an axonal conductance speed of 2 m/s, distributed conduction delays in our model (Equation 9) average just above 20 ms between these areas70–72.
In the forward path, layer 3B cells in LTM project towards STM (Figure 1). We do not draw these connections one-by-one, but as branching axons targeting 25% of the pyramidal cells in a randomly chosen MC (the chance of any layer 3B cell to target any MC in STM is only 0.15%). The resulting split between targets in layer 2/3 and 4 is typical for feedforward connections at medium distances in the cortical hierarchy69 and has important functional implications for the model (LTM-to-STM Forward Dynamics). To increase the information contrast in the forward response and balance the total current delivered to STM we also branch off some inhibitory corticocortical connections as follows: for every excitatory connection within the selected targeted MC, an inhibitory connection is created from the same pyramidal layer 3B source cell onto a randomly selected cell outside the targeted MC, but inside the local HC. This is best understood as di-synaptic inhibition via a vertically projecting inhibitory interneuron like a double bouquet and/or bipolar cell58–61. Although we do not explicitly simulate such cells, such an interneuron would be local to an MC and targeted by incoming excitatory connections (same arrangement as Tully et al. 2014, 2016). Simultaneous inputs add in non-trivial ways, as excitation and inhibition from several inputs can interfere with each other. This way of drawing random forward-projections retains a degree of functional specificity due to its spatial clustering and yields patchy sparse forward-projections with a resulting inter-area connection probability of only 0.0125% (648 axonal projections from L3B cells to STM layers 2/3 and 4 results in ~20k total connections after branching as described above.
Long feedback pathways across the cortical hierarchy are dominated by infra-granular projections (projections from large cells in layer 5 and 6), yet especially between association cortices and at short and medium range there are reciprocal projections from layer 3A cells to the cortical areas below69. In our model we draw sparse plastic connections from layer 3A cells in STM to layer 2/3 cells in LTM: branching axons target 25% of the pyramidal cells in a randomly chosen HC in LTM, again simulating a degree of axonal branching found in the literature73. Using this method, we obtain biologically plausible sparse and structured feedback projections with an inter-area connection probability of 0.66%, which – unlike the forward pathway – do not have any built-in MC-specificity but may develop such through activity-dependent plasticity. More parameters on corticocortical projections can be found in Supplementary Table 3. On average, each LTM pyramidal cell receives about 120 corticocortical connections from STM. Because about 5% of STM cells fire together during memory reactivation, this means that a mere 6 active synapses per target cell are sufficient for driving (and thus maintaining) LTM activity from STM (there are 96 active synapses from coactive pyramidal cells in LTM).
Notably LTMa and LTMb have no direct pathways connecting them in our model since plasticity of biological connections are likely too slow (LTP timescale) to make a difference in WM dynamics. This arrangement also guarantees that any binding of long-term memories across LTM areas must be the result of interaction via STM instead. Corticocortical connectivity is very sparse, below 1% total network connectivity.
Stimulation Protocol
The term Iext in Equation 1 subsumes specific and unspecific external inputs. To simulate unspecific input from non-simulated columns, and other areas, pyramidal cells are continually stimulated with a zero mean noise background throughout the simulation. In each layer, two independent Poissonsources generate spikes at rate , and connect onto all pyramidal neurons in that layer, via non-depressing conductances (Supplementary Table 2). To cue the activation of a specific memory pattern (i.e. attractor), we excite LTM pyramidal cells belonging to a memory patterns component MC with an additional excitatory Poisson spike train (rate rcue, length tcue, conductance gcue). As LTM patterns are strongly encoded in each LTM, a brief 50 ms stimulus is usually sufficient to activate any given memory.
Spike Train Analysis and Memory Activity Tracking
We track memory activity in time by analyzing the population firing rate of pattern-specific and network-wide spiking activity usually using a exponential moving average filter time-constant of 20 ms. We do not use an otherwise common low-pass filter with symmetrical window, because we are particularly interested in characterizing activation onsets and onset delays. As activations are characterized by sizable gamma-bursts, a simple threshold detector can extract candidate activation events and decode the activated memory (which is trivial due to the orthogonal and known nature of LTM patterns). We measure onset and offset by thresholding each individual activation at half of its peak rate. In LTM, we further assess pattern completion by checking for component MC activation. Whenever targeted stimuli are used, we analyze peri-stimulus activation traces. When activation onsets are less predictable, such as during free STM-paced maintenance, we extract activation candidates via a threshold detector trained at the 50th percentile of the cumulative distribution of the population firing rate signal.
Simulation Environment
We use the NEST simulator55 version 2.2 for our simulations, running on a Cray XC-40 Supercomputer of the PDC Centre for High Performance Computing. The custom-build spiking neural network implementation of the BCPNN learning rule for MPI-parallelized NEST is available on github: https://github.com/Florian-Fiebig/BCPNN-for-NEST222-MPI
Acknowledgements
This work was supported by the EuroSPIN Erasmus Mundus doctoral program, SeRC (Swedish e-science Research Center), and StratNeuro (Strategic Area Neuroscience at Karolinska Institutet, Umeå University and KTH). The simulations were performed using computing resources provided by the Swedish National Infrastructure for Computing at PDC Centre for High Performance Computing. We are grateful for helpful comments and suggestions from Drs Jeanette Hellgren Kotaleski, and Arvind Kumar.