Abstract
Recent experimental findings indicate that Purkinje cells in the cerebellum represent time intervals by mechanisms other than conventional synaptic weights. This finding adds to the theoretical and experimental observations suggesting the presence of intra-cellular mechanisms for adaptation and processing. To account for these experimental results we developed a biophysical model for time interval learning in a Purkinje cell. The numerical model focuses on a classical delay conditioning task (e.g. eyeblink conditioning) and relies on a few computational steps. In particular, the model posits the activation by the parallel fiber input of a local intra-cellular calcium store which can be modulated by intra-cellular pathways. The reciprocal interaction of the calcium signal with several proteins forming negative and positive feedback loops ensures that the timing of inhibition in the Purkinje cell anticipates the interval between parallel and climbing fiber inputs during training. We show that the model is able to learn along the 150-1000 ms interval range. Finally, we discuss how this model would allow the cerebellum to detect and generate specific spatio-temporal patterns, a classical theory for cerebellar function.
Author Summary The prevailing view in neurosciences considers synaptic weights between neurons the determinant factor for learning and processing information in the nervous system. Theoretical considerations [1, 2] and experiments [3, 4] examined some potential limitations of this classical paradigm, pointing out that adaptation and computation might also have to rely on other mechanisms besides the learning of synaptic weights. Recent experimental findings [5–7] indicate that Purkinje cells in the cerebellum represent time intervals by mechanisms other than conventional synaptic weights. We propose here a biologically plausible model which complements the modification of synaptic weights for learning one time interval in one synapse of one Purkinje cell. In the model a calcium signal in a small domain keeps track of time. Several molecules read and modify this calcium signal to learn a time interval. We discuss how this model would allow the cerebellum to detect and generate specific patterns in space and time, a classical theory for cerebellar function proposed by Braitenberg [8, 9].
Introduction
The brain ability to measure time, discriminate temporal patterns, and produce appropriately timed responses is critical to many forms of learning and behavior [10]. This is evident in the cerebellar system involvement in the learning and expression of the timing of associations by classical conditioning, such as delay eyeblink conditioning [11]. However the exact mechanism through which the cerebellum and related structures exert this function has been a matter of debate [12, 13]. Many studies have tried to locate the constituent of the cerebellar system that learns and stores the time intervals during delay eyeblink conditioning [14].
In delay eyeblink conditioning a previously neutral stimulus, the conditioned stimulus or CS (e.g. auditory cue) is paired with another stimulus, the unconditioned stimulus or US (e.g. air puff to the eye) that by itself produces a conditioned response or CR (blink). The US is presented with a certain time delay from the CS. After repeated exposure this time delay to the onset of US is learned and when the CS is presented alone (no US), the CR (blink) appears slightly before the expected US. The main circuitry for delay eyeblink conditioning seems to be comprised of (see Fig. 1): parallel fibers that might convey information about the CS (e.g. auditory cue); climbing fibers that are activated by the US (e.g. air puff to the eye); and the inhibitory Purkinje cells, the output elements of the cerebellar cortex, which seem to be responsible for gating the CR (blink). Therefore the timing mechanism might be located in the cerebellar cortex [15].
Following on this line of research, Johansson and colleagues [5] were recently led to the conclusion that the timing information for the conditioned responses is stored in the Purkinje cells of the cerebellar cortex, and that the intervals were not stored as usual synaptic weights, but as some other yet to be understood mechanism. To isolate the element storing the timing information, Johansson and colleagues [5] used direct electrical stimulation of the Purkinje cell afferents (climbing and parallel fibers), bypassing the other elements of the cerebellar system. Mimicking the delay eyeblink conditioning protocols, the electrical stimulation of the climbing fibers operated as the US; the stimulation through the parallel fibers acted as the CS; and the Purkinje cell pause in firing was the response (CR). The success of this approach is attested by the fact that after training, the electrical CS is enough to generate the CR (Purkinje cell pause in firing) at the expected delay [5].
These and other experiments show that partial stimulation of the parallel fibers (interrupted CS) still generates the CR at the same learned interval [16]. Also a change in stimulation frequency still induces the CR [17] at the expected time interval. If synaptic weights (between parallel fibers and Purkinje cell) would code for the timing, the expressed interval would depend on the drive duration and frequency. Therefore there seems to be a decoupling of the timing mechanism from further drive to the synapse. Interestingly the blockade of the main ionotropic glutamate receptors (NMDA and AMPA) did not prevent the expression of the conditioned responses or their timing [6], while blocking mGluR7 (Metabotropic Glutamate Receptor 7) prevented the timed response. mGluR7 is a G-protein coupled receptor, and hence the binding of glutamate to the receptor does not open a channel immediately, as it would be the case for ionotropic receptors, but initiates a molecular cascade that hypothetically can hyperpolarize the cell. One possible downstream effect of this cascade is the opening of Kir3 potassium channels [18]. More experiments are necessary to confirm and detail these findings, but a cascade initiated by the activation of mGluR7 is a candidate mechanism for decoupling the learning of timing from further synaptic activity.
In this work we propose a biophysical model to explain the recent observations on the learning of time intervals in Purkinje cells. In particular, we propose a model for the short time scale of the process before a long term memory is consolidated, possibly assisted by protein synthesis and non-cerebellar structures. Our model is based in known pathways invoked to explain other forms of classical conditioning [19]. For example, elements of our model such as a form of adenylyl cyclase (Rutabaga) has been shown to act as a coincidence detector and activator of the cAMP-PKA pathway in drosophila olfactory classical conditioning [20]. The same pathway also underlays classical conditioning of the gill-withdrawal reflex in Aplysia [21].
From a computational perspective the model relies only on a few steps which can be readily implemented via existent pathways in Purkinje cells. In a specific implementation CS from parallel fibers generates a calcium signal in a small domain that keeps track of the time elicited since the synaptic activation. The arriving US from the climbing fiber liberates G protein. The coincidence of calcium release and G protein liberation activates adenylyl cyclase, which in return changes the calcium signal and the activation time of calcium-dependent proteins that lead to the cell hyperpolarization. After several CS-US pairings we show how the proposed cascade makes the timing of the Purkinje cell inhibition to converge to the correct time interval.
The rest of the paper is organized as follows. In the Results section, first we describe in detail the biophysical model in terms of both the minimal computational and biophysical elements required. Secondly, we test the model in a series of numerical simulations to describe the dynamics of calcium and several proteins during CS-US pairings. The convergence to the correct time interval and the ability to learn a range of different intervals is shown numerically. In the Discussion section we compare our model to the existing literature, highlight its implications and limitations, and point out to future work. All the details for the implementation of the model and the reproducibility of the results shown here are given in the Methods section.
Results
Model
We first proceed to explain the computational properties of the model and then propose a biophysical implementation. Let us first consider a system that receives an input at some point in time. Later, through a different channel, the same system receives a second input and it should be able to store the interval between the onsets of both signals.
To store such timing information and satisfy experimental and biophysical constraints the system only needs a small set of mechanisms. First, a variable in the system should reflect the information that an input arrived and do so independently of the duration and strength of such input. To do so the system must posses a mechanism that elicits the same output even for different input strengths, and we call this element a decoupling mechanism. The output of the decoupling mechanism is a time dependent process that measures time, and we call such a process “clock”. The coincidence of an active “clock” with the arrival of a second input is registered by a coincidence detector. A final step is that the coincidence detector feeds back to the “clock” to regulate its “speed”. Positive and negative feedback loops can ensure that elements integrating the “clock” signal and acting as an output will activate at the correct timing. See Fig 2 for the proposed network of interactions able to learn time intervals.
The proposed model contains five main elements: a decoupling mechanism, a “clock”, a coincidence detector, and a positive and a negative feedback loop. These elements can be implemented by one or several molecules or alternatively one molecule can implement several of the mechanisms. Next we discuss the biophysical level of the model and identify some possible candidates for these mechanisms:
Calcium as a “clock”. Calcium is one of the most used second messengers in cells with a wide range of roles. Interestingly one of these roles is timing [22]. Multiple molecules activate depending on the spatio-temporal profile of the calcium signal [23]. In our model the integration over time of the calcium signal allows the adjustment to the inter-stimulus interval.
IP3 and calcium as a decoupling mechanism. The parallel fiber signal generates IP3 which will couple to receptors in calcium stores such as endoplasmic reticula and releasing calcium. Once released a CIRC process (Calcium Induced Calcium Release) [24] makes the calcium signal continue independently of the incoming electric signal.
Adenylyl cyclase as a coincidence detector. Adenylyl cyclase (AC) is sensitive to calcium/calmodulin and G protein [25]. The coincidence of calcium and G protein activates AC more than the sum of individual activations. This synergistic activation is also dependent on the order of arrival of the stimulus. When the calcium signal precedes the G protein the activation is more effective than in the reverse order [25]. These attributes of adenylyl cyclase account for some aspects of delay eyeblink conditioning, in which the CS has to precede the US to produce associative learning.
PKA as a positive feedback. Adenylyl cyclase activates PKA which in turn phosphorylates the IP3 receptor in calcium stores. This facilitates the release of calcium [26]. The increased release of calcium increases the activation of adenylyl cyclase and PKA.
RGS protein as a negative feedback. Some forms of RGS are activated by calcium and in turn diminish the activation of adenylyl cyclase. Thus, when RGS is increasingly activated with each pairing, the activation of AC will diminish. After RGS activation compensates the positive feedback loop, further training at the same timing will not alter the level of AC. This negative feedback loop ensures the convergence of AC levels and the timing of calcium-activated proteins.
The biochemical actors proposed in the model interact in dendrites of a Purkinje cell. The diagram shown in Fig 3 summarizes the involved molecular pathways and their interactions. In particular, input from parallel fibers causes the release of glutamate and activation of mGluR7 receptor. The activation of mGluR7 leads to the formation of IP3, which bounds to IP3R channels in the endoplasmic reticulum, releasing calcium from internal stores. The calcium signal can continue for a few seconds due to calcium induced calcium release mechanisms. Calcium alone weakly stimulates some forms of adenylyl cyclase and inhibits others. However if a second input generates stimulatory G protein (Gs), adenylyl cyclase is strongly activated. AC initiates the cAMP-PKA signaling pathway which exerts a positive feedback on the calcium signal. Finally, calcium dependent RGS inactivates G protein-coupled receptor pathways performing a negative feedback loop.
Numerical Simulations
To test the dynamics of the model and its ability to learn time intervals we simulated numerically the pathways in Fig 3. In particular, we simulated the kinetics of calcium and the involved proteins during their interactions. See Methods section for a detailed account of the kinetic models and the numerical simulations.
Model dynamics after a single CS-US pairing
In the preceding section we have seen all the elements that compose the model. Here we show how they react when a CS is paired with a US. When through parallel fibers a signal arrives at the synapse it first activates mGluR7 receptors, mGluR7 activation generates IP3 through a molecular cascade and opens intra-cellular calcium stores near the synapse. Thus the conditioned stimulus (CS) causes a sustained calcium signal as seen in Fig 4A. The US stimulus from the climbing fiber activates a receptor in the cell liberating Gs alpha protein. The coincident presence of calcium (CS) and Gs alpha protein (US) induces AC activation [25] just after US arrival as shown in Fig 4B. In turn elevated AC levels increase the synthesis of cAMP, and PKA level is observed to rise smoothly until reaching a stationary level as seen in Fig 4C. Finally, PKA phosphorylates the IP3 receptors and increases the release of calcium [26]. Thus, when tested with a second CS-US pairing, the level of calcium released by CS has increased. In our model comparing the two pairings (Fig 4B and Fig 4C) the second pairing induces around 100% more calcium release.
Learning one time interval
We have seen that a single CS-US pairing changes the activation of AC and PKA level, and increases the calcium elicited upon the arrival of the next CS. In this section we study in more detail how such dynamics allows the synapse to learn one time interval.
In our biophysical model a key to learn the inter-stimulus timing and perform the appropriate response are regulators of protein signaling (RGS). RGS proteins can vastly increase Gi alpha protein activity which inhibits AC [27]. Some forms can also inhibit Gs alpha protein production which stimulates AC, or inhibit AC by allosteric regulation. RGS proteins are inactivated by PIP3 but calcium nullifies this inactivation [28].
In the model, before pairing of CS-US occurs, RGS proteins are inactivated by PIP3 and the basal level of calcium is not enough to activate them. When the CS arrives the calcium signal will activate RGS proteins by cancelling PIP3 inactivation. During successive trainings the higher level of PKA increases the calcium signal elicited by a CS. Thus, RGS proteins will be activated earlier with each pairing, until RGS proteins are activated just before US arrival, and as a result inhibiting the Gs activation of AC. In Fig 5 we plot the dynamics of RGS response and his dependence on the number of pairings. It can clearly be seen that RGS protein responds abruptly to the input signal for the first pairing, but with a one second delay with the climbing fiber input. Posterior pairings reduce the time delay, and as shown for fifty pairings the RGS protein already anticipates the climbing fiber signal..
Learning different time intervals
We have seen how the dynamics of the molecules in the model self-organize to represent a 150 ms time interval by shifting the activation of some proteins toward the inter-stimulus interval. In this section we extend the result to a more broad range of time intervals and point out some differences between short time intervals versus longer ones.
We proceeded by training the model for several inter-stimulus intervals in the range 150-1000 ms. That is, we stimulated the model with paired CS-US inputs with different time intervals. As shown in Fig 6 after 50 pairings RGS protein learns to anticipate the climbing fiber signal for all cases.
In Fig 7 we can see the evolution of PKA during CS-US pairings at different inter-stimulus intervals. When the time interval is longer RGS proteins need a weaker calcium signal to reach the same level at US arrival. Therefore, longer time intervals reach inferior PKA levels consistent with smaller calcium signal.
Discussion
Summary
Recent experimental results in delay classical conditioning [5–7] indicate that some form of time interval learning takes place inside the Purkinje cells in the cerebellum. Here we have developed a biophysical model for time learning in dendrites of the Purkinje cell. The model relies on a few computational steps that can be readily implemented by pathways invoked to explain other forms of classical conditioning and are present in the dendrites of Purkinje cells. In the model parallel fibers input (CS) elicits a calcium signal followed by climbing fiber input (US) activating G protein coupled receptors that liberate Gs protein. The coincidence of calcium and Gs protein activates adenylyl cyclase, which via a PKA cascade changes the calcium signal and the activation time of calcium-dependent proteins. After a few CS-US pairings, the activation time of calcium-activated proteins learns to peak at the interstimulus interval and the Purkinje cell can pause at the correct time.
The general idea proposed here is that, regardless of the specific molecules involved, molecular pathways inside dendrites of Purkinje cell possess the necessary machinery to learn time intervals. Indeed the computational steps leading to time interval learning and explaining recent experimental observations are simple, and only involve basic interactions such as a saturating response, a coincidence detector and a negative feedback loop.
Relation to other models of time interval learning in Purkinje cells
The model presented here has similarities to the model proposed by Steuber and Willshaw [29], both using a calcium signal acting as a memory trace to link the timing between two events. Also the level of phosphorylation controls the calcium signal and the delay. However their model is very similar to classical synaptic plasticity mechanisms since it proposes that the time learning modifies the availability of mGlu receptors. Therefore their model lacks a mechanism that decouples delay timing from electric strength which is in contradiction with experimental results. Our model considers the number of receptors fixed, thus if their response saturates for small short signals the delay is independent of electric strength and duration.
Including the lack of decoupling mechanism Johansson and colleagues [17] pointed out several more shortcomings of the model proposed by Steuber and Willshaw:
The model learns for short interstimulus intervals(<100 ms) in contradiction with experimental data [30]
If the interstimulus interval is changed in the model the pause response will move gradually to the new time. Experimentally the old time response disappears and a new response appears without passing through intermediate delays.
A Purkinje cell trained with two interstimulus intervals generates two responses. The model is only able to generate one response not two responses for two different times.
In our model the first problem is absent due to AC activation profile which prevents the learning of very short time intervals. The activation of AC is highly dependent on the temporal ordering of stimuli: if the Gs protein (elicited by US) is released before calcium (elicited by CS) or simultaneously the response of AC is small. Given that it takes some time for the calcium signal to start, for short interstimulus intervals the learning will not take place.
In line with Jirenhed et al. [31] we would also like to suggest a simple explanation to avoid the second and third shortcomings; given that the Purkinje cell might have around 100000 synapses the learning can take place at many synapses in a stochastic manner. That each single spine of the Purkinje cell constitutes a computational unit was already proposed by Den et al. [32]. Different synapses can learn different times producing more than one timed response. Also the transition to a new response will not pass through intermediate time intervals because new synapses will learn the new time meanwhile the old time is erased from others.
The capability to learn at multiple dendrites offers additional advantages. For example, in a natural situation the Purkinje cell would have to recognize sparse incoming sequences. The ability of a Purkinje cell to learn timing information at multiple synapses helps to integrate evidence arriving at different times and produce appropriate responses. In other words, even though neurons could be modeled with a single delay mechanism that learns more than one time to match the experimental evidence obtained in Purkinje cells, it is more likely that in the natural conditions the synapses would have to be timed independently to produce the behavioral response at the adequate time.
Ricci, Kim, and Johansson proposed an alternative model [33]. In that model the cell releases recording units that evolve stochastically and later for each one of them generates a small inhibitory current. The model does not explicit a concrete biological implementation but reproduces well the experimental results. Interestingly the output sum of currents by noisy recording units is equivalent to sum noisy times at each synapse. Thus the same replication of experimental results will be obtained for a collection of noisy timing synapses. Moreover the later is able to explain better other cerebellum functionality beyond delay eyeblink conditioning as we suggest below.
Implications for Braitenberg theory of cerebellar function
The cerebellar anatomy seems well understood, to a first approximation it consists of a single circuit that repeats itself [34]. However many functions have been assigned to the cerebellum [35] i.e.: timing of classical conditioning [36]; prediction of stimuli [37]; error correction [38]; forward models in motor control [39] and motion compensation in perception [40]. The lack of a single function for the cerebellum seems to be at odds with its structural uniformity. However, some proposals suggest that its structural uniformity reflects the computations performed by the cerebellum [41], even if the higher level functions that exploit these computations look radically different.
One of the first theories for cerebellum function is the proposal by Braitenberg [8], in which the cerebellum responds specifically to certain time sequence of events in the input and learns to generate other specific sequences of signals in the output. Originally, it was proposed that the key to achieve this computation was the existence of different delays or latencies of the input to Purkinje cell, but given the lack of delay lines in cerebellum circuitry Braitenberg modified his hypothesis [9]. In particular, the lack of experimental evidence of delays in cerebellum circuitry made very difficult to justify how the cerebellum might integrate past information to learn time sequences. For example during arm movement several muscles move at different times and the cerebellum is able to predict the final position of the hand. The movement of each muscle affects the final result in a nonlinear way so its necessary that all the combined information is made available at the same time to predict the position.
However, it is interesting to consider Braitenberg old hypothesis given experimental findings in delay eyeblink conditioning [5]. It is possible that delay eyeblink conditioning can be reproduced with the Purkinje cell learning time intervals at the soma, but to explain other cerebellum functions the Purkinje cell has to be able to integrate time sequences. If each Purkinje posseses multiple timing mechanism along the dentritic tree, it will allow the cell to respond to complex spatio-temporal inputs with any desired output. Normally there will be a cause-effect relation between inputs and output: the output might be a motor action to avoid or seek the effects of the inputs, or the effects themselves to be processed in another brain region (one possible exception is that the output is the timing between two inputs). The main limitation of this approach is the time between input and output, the further away in time the more difficulty is to model cause-effect given that the possibles causes are all the combinations of all the inputs received. Thus for learning to predict there is a trade-off between how much time into the future the prediction goes and the number of possible causes taken into consideration. Cerebellum seems well posed to favor the later, that is, processing sensorimotor information and making predictions in the millisecond range.
Model Limitations
Our main contribution is the proposal of a computational model able to self-organize to learn appropriate time intervals. However, we also explicit out a possible molecular implementation of the model. We believe there are many alternatives to such a implementation and indeed here we discuss several limitations of our own specific proposal.
Many parameters of the model have been obtained from in vitro studies. Typically in vitro assays for biological reactions are studied in equilibrium and in controlled conditions. However biological pathways might depend on the dynamics of the signals [42] and the convergence of multiple interactions. In our case AC activation might depend on the timing between CS-US, this dependence can differ greatly from the model, depending on the conditions inside the cell.
Recent experiments indicate that in a group of Purkinje cells that predict saccadic movements, half of the cells might produce increased activity at precise time instead of pauses [43]. It remains unclear if the timing mechanisms are the same in the two groups.
Here we have modeled that the US arriving from climbing fibers activates G protein coupled receptors (GCPR) without entering in detail. Given that the climbing fiber innervates all the dendritic tree, it might also activate receptors in a more direct manner to elicit the learning mechanism. Another possibility is that the pause in firing activates the receptors. However, optogenetic essays in the Purkinje cell [44] show that a pause alone does not suffice to learning the timing: instead when increased excitation in Purkinje neurons precedes a pause (without teaching signal from climbing fiber), the timing is learned. It is possible that when a climbing fiber signal arrives, the high frequency burst of somatic spikes activates a retrograde chemical signal that activates GCPR along the dendritic tree.
We have modeled interval timing with a calcium signal. Although it is known that calcium signals and oscillators can reflect timing information, is not clear how the information is contained in the spatio-temporal profile of the calcium levels. To make the model simpler we have modeled activation of RGS proteins almost instantaneous after certain level of calcium, but this might differ greatly in reality.
Future work
We have proposed a mechanism for interval timing in the cerebellum. Future experimental work has yet to elucidate if the Purkinje cell is able to record the time between stimulus at synaptic level independently of synaptic strength. One first step will be to train two independent group of synapses each one with a different time, and after training check if the cell responds with a different timed response depending on the group of synapses stimulated.
The model here presented is simple compared to the molecular mechanisms that process information in cells. It is probable that protein translation and even epigenetic mechanisms play a role in the storage of the memory. Along with experimental work, we hope that different models will be proposed, modified, and compared to explore several potential intra-cellular mechanisms of time interval learning. We believe this can help to guide experimental research to finally elucidate how Purkinje cells store timing information.
Conclusions
We have shown that is possible to learn time intervals at one synapse without classical synaptic plasticity. Although we expect the real mechanism to differ, we believe that our model is biological plausible and serves to illustrate that a few computational steps can perform time interval learning inside Prukinje cells. The model is based in the cAMP-PKA pathway involved in some forms of classical conditioning. In the model a calcium signal keeps track of time like in other forms of biological timing, and the “speed” of the signal is adjusted via feedback loops until calcium-dependent proteins activate at the correct time interval.
The capacity to learn time intervals at each one of the synapses in the Purkinje cell might validate Braitenberg theory: the cerebellum responds to certain time sequences of events in the input and learns to generate other specific sequences of signals in the output. Future experimental evidence might help to confirm the bridge between molecular mechanisms at microdomains and psychological function in the cerebellum.
Methods
Simulations were performed in the NEURON environment (Version 7.4) [45], with backward Euler integration method and 25 µs time steps. The simulations where executed at the Comet supercomputer at the San Diego supercomputing center through the Gateway Portal [46].
Calcium Store
The calcium oscillations follow the Somogyi-Stucki Model [47]: where x is the calcium concentration inside the pool and y in the cytosol.
When there is no parallel fiber stimulation the calcium at the synapse is at certain basal level:
PKA phosphorylation induces at least a fourfold increase in the sensitivity of InsP3R1 to activation [48]. Therefore when a parallel fiber stimulus arrives we initiate the value for the calcium inside the pool at 25 (x=25), and for two seconds the calcium at the synapse follows the formula:
Where [PKA] is the level of active PKA and [Ca2+] the calcium released.
PKA Pathway
Active adenylyl cyclase is given by the following equation:
To take into account the synergy between calcium and g protein to activate adenyl cyclase we have taken the following rates:
The other equations for the pka pathway are from the model of cAMP dynamics by Viole et al [49].
RGS protein
The evolution of RGS protein is given by:
The rate is given by the following formulas
G protein-coupled receptor
As soon as a climbing fiber input arrives during 200 ms generates the following quantity of g alpha protein:
Author contributions
D.M. implemented and performed all computational work and wrote the main manuscript text. A.Z. discussed the results and wrote the main manuscript text. R.V. directed the project and wrote the main manuscript text. All authors reviewed the manuscript. All authors read and approved the final manuscript.
Acknowledgments
R.V. and D.M. thank the financial support from the Estonian Research Council through the personal research grant PUT1476. This work was supported by the Estonian Centre of Excellence in IT (EXCITE), funded by the European Regional Development Fund. The authors also thank the Gateway Portal for the computing resources provided [46]. The authors are indebted to Luiz Lana for writing a draft of the introduction, in depth discussions leading to the proposed mechanism and many ideas here discussed. Likewise the authors thank Jaan Aru, Andres Laan and Juhan Aru for early discussions.