Abstract
We construct a model of cell reprogramming (the conversion of fully differentiated cells to a state of pluripotency, known as induced pluripotent stem cells, or iPSCs) which builds on key elements of cell biology viz. cell cycles and cell lineages. Although reprogramming has been demonstrated experimentally, much of the underlying processes governing cell fate decisions remain unknown. This work aims to bridge this gap by modelling cell types as a set of hierarchically related dynamical attractors representing cell cycles. Stages of the cell cycle are characterised by the configuration of gene expression levels, and reprogramming corresponds to triggering transitions between such configurations. Two mechanisms were found for reprogramming in a two level hierarchy: cycle specific perturbations and a noise induced switching. The former corresponds to a directed perturbation that induces a transition into a cycle-state of a different cell type in the potency hierarchy (mainly a stem cell) whilst the latter is a priori undirected and could be induced, e.g., by a (stochastic) change in the cellular environment. These reprogramming protocols were found to be effective in large regimes of the parameter space and make specific predictions concerning reprogramming dynamics which are broadly in line with experimental findings.
I. INTRODUCTION
The retrieval of pluripotent cells was first pioneered by John Gurdon in the 1960s, using nuclear transfer to clone a frog1. More recently, cell reprogramming has shown that it is possible to obtain induced pluripotent stem cells (iPSCs), which strongly resemble embryonic stem cells (ES), from somatic cells via the introduction of just 4 transcription factors (Oct3/4, Sox2, Klf4 and c-Myc - now known as the Yamanaka or OSKM factors)2,3. It has also been demonstrated that nearly all somatic cells can be reprogrammed in this manner4, suggesting that the “code” for pluripontency lies in the genome common to all cells of an organism. Once reprogrammed it is possible to guide iPSCs to differentiate into a desired cell type using specific culture conditions5. Due to their ability to self renew and differentiate into many different cell types, stem cells (including iPSCs) hold great potential for both personalised and regenerative medicine6. Furthermore, iPSCs can act as a model environment for studying disease and testing drug delivery mechanisms7,8. Since the original reprogramming experiments, multiple protocols have been uncovered by replacing certain Yamanaka factors with other proteins or small molecules. For an extensive biomedical review of iPSCs, and how they differ from other stem cells, see Takahashi 20159.
However, despite the great potential of iPSCs, and the evolution of cell reprogramming experiments, much is still unknown about the decisions governing the fate of a cell. Cell fate decisions were first modelled by Waddington using his idea of an epigenetic landscape. This model describes development with the analogy of a ball rolling down a hill from states of high potency to fully differentiated ones. Different cell types are represented as valleys in the landscape, and a cell's fate is determined by the valley which the ball falls into10. The number of valleys increases the further the ball moves down the landscape, representing the increasing diversity of cell types during development. Whilst this model provides an interesting metaphor for differentiation, it lacks some key aspects of cell biology, such as cell cycles. Recent experimental work has suggested that cell types can be considered as high dimensional attractors of a gene regulatory network11, paving the way for a dynamical systems approach to cell reprogramming.
Many of the current models describing cell fate decisions focus on specific small gene regulatory networks (GRN) that are believed to govern pluripotency or differentiation, or approach the problem from a cell population perspective. For further details of the current mathematical and computational models of cell reprogramming the reader is directed towards the references Morris et al. 201412 and Herberg and Roeder 201513 respectively.
In this paper a theoretical model is presented which models cell reprogramming in terms of transitions between attractors of a high dimensional dynamical system, describing the transcriptome of a cell. The attractors of the dynamics represent cell cycles of different cell types, which are related to one another in a hierarchical manner.
The rest of this paper is organised as follows: In section II the model is formulated by appealing to a small set of key observations concerning cell chemistry, before being applied to a specific type of hierarchy in section III. In section IV evidence for reprogramming in the model is presented and discussed, with the main findings of the work summarised in section V. The majority of the mathematical details of calculations have been relegated to the appendices in aim to make this article accessible to readers from various backgrounds.
II. THEORY
Cells are the fundamental units of structure and re-production in most organisms14. They are complex and dense building blocks which contain a rich tapestry of biochemical reactions involving a multitude of chemical species (e.g. proteins, sugars, lipids, etc). Metabolic pathways, such as glycolysis, involve many intermediate steps converting the product of one reaction into the substrate for another. Enzyme reactions, like those involved in gycolysis, can be described generally by a set of differential equations, known as the Michaelis-Menten equations15. Thus, to fully describe the dynamics of cell chemistry one would need to incorporate the MichaelisMenten equations for all possible reactions into a theory which would describe reaction and diffusion mechanisms, self organisation, biochemical signalling, etc. One component of such a theory would be the transcription of the N genes of the organism’s genome (∼ 25,000 for humans), which alone represents a vast state space. For example, even if one assumes binary gene expression levels (i.e. genes are either expressed or not expressed), there are 2N possible configurations of gene expression levels. A natural question then arises from the complex chemistry of cellular life: How do so many reactions, of so many species, give rise to a comparably low number of different cell types? For example, the human genome (common to all cells in an individual) is comprised of approximately 25,000 genes yet only gives rise to around 300 different cell types. A plausible realisation of this fact is to suppose that stable cell types emerge as attractors of the full reaction dynamics of the cell.
For the purpose of modelling cell reprogramming we propose to construct a reduced model by appeal to the following line of reasoning. Suppose one was able to in-tegrate out all components of the complete theory other than gene expression levels, the result would be a reduced model which will have the following two features: (i) it will involve interactions between genes; (ii) the interactions will exhibit memory effects. The interaction of genes would result in a feedback mechanism that could explain the existence of stable attractors. In the reduced model, memory would be a result of the interplay between genes and proteins. Transcription factors are proteins that regulate the expression of genes (through ac-tivation/inhibition). These proteins are translated from RNA, which is transcribed from the genes in the cells nucleus. Thus, the expression level of a gene will depend on the previous expression levels through gene regulation. Furthermore, proteins can regulate the genes which they were synthesised from, other genes, and/or combine with other proteins to form complexes which are transcription factors and hence the expression level of a given gene will depend on the previous expression levels of many (or all) genes. Memory is in fact required to create dynamic cell cycle attractors with different durations for each of the phases of the cell cycle in a model based on gene expres-sion levels only.
Based on these observations, we build a minimal model that describes cell types in terms of gene expression levels across their cell cycles. We can make simplifications to the reduced model, that do not change the intuition behind, or nature of, the model but make the mathematics easier to work with. One such simplification is the discretisation of time, which allows one to neglect the effects of memory. To do this we measure time in terms of stages passed through the cell cycle (e.g. G1, S, G2,…). This allows one to ignore the different durations of each cycle phase by concentrating on which phase of the cycle a cell is in. Another assumption is that the gene expression levels are binary variables, ni (with i = 1…N), i.e. genes can exist in one of only two states: they are either expressed or are not. These states may be represented by the binary values ni = 1 and ni = 0 respectively, hence the common terminology Boolean, or “on/off”, genes. Again, it is important to stress that these assumptions make the mathematics of the model much simpler, but can be relaxed if a more comprehensive description of cell cycle regulation is re-quired.
A general model for the dynamics of interacting binary genes would have the following form, where ni is the gene expression level of the ith gene, with the effect of the gene interactions encoded in a local field,hi(t) of the form,
Here Jij is the effect of the interaction between genes i and j, and Jijk is likewise the effect of the triplet interactions between the 3 genes i, j and k, (there could also be higher order interactions which are represented by the…). Any constant contributions to the local field, such as self regulation, can be absorbed into the definition of θi. The ξi are random variables with zero mean and a suitably normalised variance, which mimic noise to represent the fundamental stochasticity of reaction events. Popular noise models are Gaussian and thermal noise. We use T to vary the strength of the noise. Anticipating our later choice of the thermal noise model, we will refer to T as to temperature. The Θ[x] is the heaviside step function: Θ[x] = 1 for x ≥ 0 and Θ[x] =0 otherwise. Thus, (1) states that a gene will be expressed in the next phase of the cell cycle (i.e. italic>ni(t + 1) = 1) if the combined effect of all interactions and stochastic noise exceeds a gene specific threshold, θi. At each time step every gene expression level is updated according to this rule, and the state of the system is fully described at any time t by the instantaneous configuration n(t) = (n1(t),…nN(t)) of gene expression levels.
A. Minimal model
We restrict ourselves to consider a system involving pair interactions only, and simplify matters further by assuming uniform thresholds, i.e. θi = θ ∀ i. Thus the dynamics of the minimal model is given by the following simple expression,
This expression is reminiscent of the models used in the field of neural networks (NNs) for associative memory, with a post synaptic potential (PSP), hi =Σj Jijnj and a neuron fires (ni(t + 1) = 1) if the PSP exceeds a given threshold θ. In associative memory, configurations of neuronal activity representing some memories are stored in the synaptic efficacies Jij, such that they are attractors of the dynamics. The NN is then said to recall a pattern when the system converges to the corresponding configuration from some initial condition. Such NNs are said to be content addressable because the attractor to which they converge is given by the (content of) the initial state. Associative NNs of this type are robust to input errors and hardware failures (such as disruption of the synaptic efficacies and thresholds). Analogously, in our model, specific configurations of gene expression levels, which represent the different cell types of an organism, are stored in the gene interactions, Jij, which therefore govern dynamics. Using a temporal ordering of the cycle state specific configurations, the attractors become dynamic attractors that represent the cell cycles of each cell type (see section IIC for details). It is also desirable that the gene interaction network is robust to variation or errors in gene expression levels, because despite variation in gene expression levels across human individuals and mutations in the human genome, indi-viduals of the population have the same set of cell types.
Hanna at al. demonstrated that nearly all cells can be reprogrammed4. This suggests that the “code” for pluripotency lies in the genome that is common to all cell types of a given organism. The analogy between associative memory and cell reprogramming has been made recently in the context of protein interaction networks16. However, that work differs from that presented here due to the absence of cell cycles and the potency hierarchy. In their work, Lang et al. extend Waddington’s developmental landscape metaphor into an epigenetic landscape describing the interactions between proteins. On the other hand, by working with gene interactions it is possible to neglect the specific activation/inhibition nature of transcription factors, which will naturally arise from the interaction between genes. Thus, the authors feel that the interaction between genes, as opposed to proteins, is a more natural approach to model cell fates.
B. Cell cycle similarities, lineages and reprogramming
Cell reprogramming requires transitions between cell fates, which can be either a trans-differentiation or a de-differentiation, i.e, either across or up the the potency cascade9. With this idea in mind the cell cycles stored in the model are related to one another in a hierarchical manner. Specifically - apart from the stem state which sits at the top of the hierarchy and thus has no ancestor (see figure 1) - the gene expression levels of all cell types are conditionally dependent on their parents. This set up is inspired by the storage of memories, in a Markovian hierarchy, for associative memory NNs17. Parallels between the hierarchical relation of cell cycles and Waddington's epigenetic landscape can be made. However, the present approach does not directly model a landscape of the cell states.
It has been demonstrated that protein and mRNA levels vary across the cell cycle18, thus it is reasonable to infer that the gene expression of a cell type also changes throughout its cycle. It is thus plausible to conceive of a situation in which the global expression levels of different cell types are more similar in certain stages of the cell cycle than in others. For example, during the S-phase the gene expression levels could likely be vastly reduced in all cells types (as suggested by Cho 199819), as the DNA is otherwise occupied through replication. It is also the case that most of an organisms cells undergo the mitotic phase via broadly the same mechanisms. Hence, there could exist (at least) one phase of the cell cycle in which different cell types are more similar than others (see figure 2). These stages of the cell cycle would represent a natural target in which it is easier to induce switches between cell types, i.e. to reprogramme a cell. This is one of the main hypotheses that we will be testing in the present study.
The authors are aware of only one other model which includes a hierarchy of cell states and the cell cycle. In their model, Artyomov et al. defined a cell type through the expression levels of a small ensemble of master regulatory genes referred to as a module20. They included the cell cycle as an interplay between the gene expression levels of a cell and the epigenetic state of the cell. On the other hand, we treat each cell type as a dynamic entity, which transitions through different configurations of gene expression levels corresponding to stages of its cell cycle. Each configuration described the entire transcriptome of a cell in a given cycle phase. In this work we do not directly model the epigenetics of a cell. However, the similarity between different cell types, during specific phases of the cell cycle discussed above, could be a result of epigenetic changes, such as changes in chromatin structure or the presence of histone markers.
C. Two-level hierarchy
To validate the principles of our approach we apply the neural network model above to a simplified version of the biology. This makes the mathematics easier to implement and keeps the notation simple and transparent. Such simplified scenarios still capture the main principles of the biology, and in practice the mathematics can easily be extended to more realistic systems. We therefore consider a two level hierarchy in which fully differentiated cells are direct descendants, or daughters, of the stem cell (see figure 3). Each of cell cycles are coarse grained into 3 stages, with a single cycle phase made more similar across the different cell types. The coarse graining of the cell cycles was carried out purely for computational efficiency and generalisation to 4 or 5 stage cell cycles is straightforward.
III. PROBABILISTIC FRAMEWORK
We will now introduce a specific model which implements our generic reasoning within a probabilistic framework. Consider a system of N genes, each labelled by i = 1…N, and M daughter cell types, labelled by μ = 1…M (for a full list of mathematical notation see table I). Each cell type, daughter or stem, undergoes a cell cycle of length C and we denote each phase of the the cycle by ρ =1…C (with C +1≡ 1). The expression of the ith gene, in the ρth phase of the stem cell cycle, is denoted by , where corresponds to that gene being expressed. We denote by aρ the fraction of genes expressed during the ρth cell cycle phase, also referred to as the activity of that cycle phase. Thus, the probability of the gene expression levels in the ρth phase of the stem cell cycle are given by
Then the configuration of the stem cell state, in the ρth cycle phase, is then given by ηρ = . Furthermore, for every state, ηρ, of the stem cell cycle there is a corresponding set of descendants, in the same stage of the cell cycle, ηρμ = .Similar to the stem cell cycle, each daughter cell μ has an activity given by aρμ, which governs the probability of expressing a gene in each stage of the cell cycle
We assume the gene expression levels in the stem cell, , are independent, identically distributed (i.i.d) random variables, which implies that the configurations ηρ are independent along the cell cycle. This assumption was made to simplify the mathematics, but may be relaxed if a more comprehensive model is desired. The configurations of the daughter cells, on the other hand, are derived from the corresponding phases of stem cell. We define the probability of turning a gene off during the differentiation-transition from a stem cell to a daughter cell, in the same phases of the cell cycle, with equal activities aρ = aρμ, as γρμ. Thus, the transition probability of a gene being expressed in both the stem and daughter cell (of the same cell cycle phase) is given by , i.e. the ratio of probabilities the gene is on in both states multiplied by the probability it was not turned on in differentiation. The full transition matrix from the stem cell state to a daughter cell cycle state in the same phase can be found in appendix A. Due to this construction of the daughter cell cycles in terms of the the stem cell cycle, the different daughter cell configurations are conditionally dependent on the stem cell state.
The interactions between the gene expression levels of the system should be chosen such that the cell cycles of daughter and stem cells are attractors of the dynamics. To construct the interactions of the model we combine a set of known results from the field of NNs. Hopfield originally showed that multiple configurations can be stored in the synaptic couplings of a neural network using the Hebb rule21. It is known that dynamic attractors can be stored in the couplings by adapting the Hebb rule to include a temporal order to the stored configurations, i.e the interactions have a contribution from the current pat-tern and its successor22,23. Thus, in our notation of gene expression levels, a sequence of stem cell cycle phases may be stored in the interactions in the following manner.
This choice of interaction ensures that if the gene expres-sion levels evolve according to (3) and are initialised, at time t, in the configuration n(t) = ηρ, their configuration in the next time step will be n(t) = ηρ+1 for sufficiently low noise levels. To ensure the sequence of configurations retrieved by the system is a closed cycle, the successor of the final configuration must equivalent to the initial con-figuration. For low activity configurations it is required that one removes the bias from each of the cycle phases, in order to achieve stable limit cycle attractors in the dynamics. Here this is done by subtracting the average gene expression of each of the cell cycle phases, resulting in the contributions from each cycle phase having zero mean.
Information can also be stored in the interactions in a hierarchical manner17,24,25 (equivalent to the structure shown in figure 3). This is done by including contributions from each statine in the hierarchy in the interactions. Each pattern must be weighted by a factor determined by its position in the hierarchy26. Combining these two ingredients, the interactions that stabilise a hierarchy of cell cycles can be written as follows,
Here the summations are over cycle phases, ρ, and daughter cell types, μ. We chose to remove the bias from the daughter cell type by subtracting the conditional average of the gene expression, aμ(ηρ) = 𝔼 [ηρμ|ηρ], i.e. the average gene expression level of the daughter cell given the expression levels in the same cell cycle phase of the stem cell. However, the bias could also be removed using the activity of the daughter cells, aρμ, in place of the conditional averages in (7). The weights in the denominators are the variances of the gene expression levels in the corresponding cell cycle stage for the stem or daughter cells. Note that, if the ρ +1 was replaced with ρ in (7) then this would be the standard prescription for storing a hierarchy of configurations. Since they are included we have in fact stored a hierarchy of cell cycles.
Given the form (7) of the interactions, one can express the local fields hi(t), appearing in the dynamics (3), concisely in terms of a set of macroscopic dynamical order parameters, namely giving,
Here we have now absorbed the gene specific threshold θ, appearing in 3, into the definition of hi(t). The value of θ is fixed, such that the stable cell cycle attractors exist at sufficiently low noise levels. Note that hi(t) = , in which is the vector containing the set of all dynamical overlaps and . As a consequence, and by appeal to the law of large numbers in the limit of a large number N of genes, one can formulate the dynamics of our model in closed form entirely in terms of the dynamic order parameters, giving provided that N ≫ M in this limit. In (11) and (12) β = 1/T is the inverse of the noise strength. The 𝕡(ξ ≤ z) in (11) and (12) is the cumulative distribution function (CDF) for the noise probability, P(ξ) (i.e. the probability that the ξ will take a value less than or equal to z). Popular choices for the P(ξ) are the Gaussian distribution, and the qualitatively and quantitatively similar, thermal distribution . We will use the latter, for which (11) and (12) can be written in the following form (for details of this calculation see the appendices B), where the angle brackets, 〈…〉ηρ,ηρμ, represent the average and conditional averages over all stem and daughter cycle states. These equations of motion are easily solved numerically by forward iteration, starting from suitable initial conditions.
IV. RESULTS
In the results that follow a single cell cycle phase was made more similar between the daughter and stem cell cycles. This was done by having a lower value of aρ and aρμ in that particular phase. The similarity between the the cycle phases of the stem and daughter cells can be seen from the covariance between their expression levels,
Thus the probabilities of expressing a gene in both cell cycle stages (aρμ and aρ respectively) and the probability that a gene is not turned on during differentiation (i.e. 1 −γρμ) govern the similarity between the gene expression levels of the same cycle phase across different cell types. Thus, it is possible to make certain stages of the cell cycle more similar across the two levels of the hierarchy by tuning the parameter values used in (15). At this point it should also be noted that there are restrictions on the values that γρμ can take, in order for the transition probability from ηρ to ηρμ to be correctly defined as a probabilities (for details see the appendix A).
Unless stated otherwise, the following parameter values will used in all of the results and analysis. Firstly, aρ = 0.7, aρμ = 0.6. These values were chosen based on the analysis of the number of genes expressed in Ramsköld 2009, which suggests that 60 — 70% of all genes are expressed in human cells27. Then for the cycle phase which was designed to have a higher correlation across different cell types we use aρ = 0.3, aρμ = 0.2. The motivation for these values to be lower than in other phases, arises from the hypothesis that the maximally similar cell cycle stage is due to the genome being occupied with other processes (e.g. DNA synthesis during the S phase). The value γρμ = 0.2 was used for all cell cycle phases, ρ, and daughter cell types, μ, and the threshold values were set to zero (θ = 0).
For the presentation of the results we use the so-called overlaps, which measure the correlation between the state of the system n(t) and the gene expression patterns that are characteristic of the cell cycle states of the stem and daughter cells, respectively. They are closely related to the dynamic order parameters, and in fact identical for the stem cell cycles, and are defined as,
The overlaps are normalized to to have values in the interval [–1, 1]. An overlap of mρμ = 1 (or –1) means the system is fully correlated (or anti-correlated) with the cell type μ;, in the cell cycle phase ρ, whereas mρμ = 0 implies that they are completely uncorrelated. The same is true for the stem cell cycle and corresponding values of mρ.
In figure 4 we plot the numerical solutions of (13) and (14) when the system is initialised in a daughter cell cycle, at a low noise level, T = 10−2. Peaks of the dashed line correspond to the system transitioning through the states with a high overlap with the daughter cell cycle phases mρμ. Similarly peaks in the solid line correspond to the overlaps with different phases of the stem cell cycle, mρ.
Because of correlations between the patterns of the cycle states of stem and daughter cells, one observes nonzero mutual overlaps between them. Specifically if the system is in a (perfect) daughter cell state, , the overlap with the corresponding stem cell state is
Conversely if the system is in a stem cell state, , the overlap with the daughter cell state is
The ρ = 2 phase of the cell cycles (for both stem and daughters) use parameter values that give rise to a higher covariance between the stem and daughter cell types. This can be seen from the increase in overlap every 3 time steps in figure 4. The other two phases of each cell cycle have the same values of activities for their given cell types. The initial value of mρ(n(t)) used in figure 4 was determined using (18).
A. Noise induced switching
At a low noise level, if the system is initialised in a daughter cell it will transition along that cell cycle in-definitely. However, as T is increased above some critical value the noise will take the system away from the daugh-ter cell and it will fall into the attractor corresponding to the stem cell cycle. If the noise is then removed completely the system will become fully correlated with the stem cell cycle. The noise induced transition from the daughter cell cycle to the stem cell cycle is shown in the left panel of figure 5 for a value of the temperature T at which the daughter cell cycle is no longer stable. Monte Carlo simulations of the dynamics for N = 25, 000 confirm the validity of our analytic solution formulated in terms of the macroscopic dynamic order parameters in (13) and (14) - right panel of figure 5. In this figure we do not plot all time dependent overlaps as in figure 4 but only the “envelope” of the overlaps defined as the overlaps mρ(t) and mρ(t)μ with the expected cycle state, given by ρ(t) = 1 + (t mod C).
The de-differentiation transition takes multiple time steps before a steady state is reached when the system is in the stem cell cycle attractor. This kind of dynamics is in line with that seen in reprogramming experiments, which take multiple generations of the cell cycle before the iPSCs strongly resemble embryonic stem cells3,4.
If, however, the noise level is too high the system quickly loses any correlation with all cell cycles - i.e. all the overlaps become zero. To find the range of noise levels over which it is possible to retrieve the stem cell from the daughter cell cycle one can investigate the stability of the solutions of (13) and (14). Alternatively, we carried out the following numerical experiment: the noise level, T, was incremented from zero and at each T the equations of motion were solved numerically, the steady state values of the overlaps (mρ(t) and mρ(t)μ) which the dynamics converged to were then recorded. Theses steady state values are plotted against the corresponding noise level in figure 6. It is clear that above some critical T reprogramming via de-differentiation to the stem cell occurs due to the noise in the system. The value of this critical T will depend on aρ, aρμ and γρμ.
B. Direct perturbations
The noise induced de-differentiation is different from reprogramming experiments, in which the dedifferentiation is due to a direct perturbation using factors that are common to embryonic stem cells (i.e. the Yamanaka factors). Such a directed perturbation can be modelled in our system by introducing an extra contribution λi(t) to the local field, which pushes the system in the direction of the stem cell cycle, and has the form
Here k is the strength of the perturbation, ρ̅ is the stage of the cycle to which the perturbation is applied, and ci is a logical variable representing whether or not the perturbation is applied to gene i (ci = 1 with probability q, and 0 otherwise).
Since one of the phases of the cell cycle is more similar across different cell types, it is an obvious target for per-turbations when attempting to reprogramme a cell. The perturbations should be applied just prior to the most similar phase so as to only minimally disrupt the progres-sion of the cell cycle. So choosing ρ̅ as the cycle phase prior to the maximally similar one, is expected to be the optimal reprogramming protocol at a given temperature.
In figure 7 we are carrying out the same numerical ex-periment as in figure 6, except the probability, q, that a perturbation is applied, is incremented rather than the noise level, T. This experiment shows that dedifferentiation is possible with a directed perturbation even at low noise levels where the daughter cell cycles are stable. The retrieval of stem cell cycle is only possible above some critical value of the fraction of perturbed genes, that we call the reprogramming threshold, qr. Because the ρ = 2 cell cycle stage is more similar across different cell types, perturbations applied to ρ̅ = 1 should have a lower qr value compared with perturbations ap-plied to other phases, i.e. ρ̅ = 2 or ρ̅ = 3. This is indeed borne out by the theory.
Increasing the noise level towards the critical T required for noise induced de-differentiation can dramatically change qr, see figure 8. The critical value qr has a non-monotonic dependence on the noise level, T. This is a direct result of the non-linear nature of the system and the dependence of the perturbation (19) on the dynamical order parameters. As expected the ρ̅ = 1 perturbations have the lowest qr values at any given T. This is because the ρ = 2 phase was made to exhibit the largest degree of mutual similarity among cell types, due to a decreased activity in this phase. The fact that the ρ̅ = 3 perturbations have a lower qr than the ρ̅ = 2 perturbations follows the Hamming distance between the state in which the perturbation is applied and the stem cell state targeted by that perturbation, which is smaller for a perturbation applied in the 3μ state than for a perturbation applied in the 2μ state. That is, d[η3μ, η1] < d[η2μ, η3], where the normalized Hamming distance between states is defined as
Hence a higher fraction of genes need to be perturbed to achieve de-differentiation using ρ̂ = 2 compared with ρ̂ = 3.
We have also looked at a case where perturbations of the form (20) are acting during multiple stages of the cell cycle in the reprogramming experiments. For example during the most similar phase and the one prior to it. In the case effects of each perturbation are combined and qr may decrease compared to applying perturbations to a single phase - right panel of figure 8.
V. SUMMARY
In this paper we presented a general (minimal) model for cell reprogramming as transitions between attractors of a dynamical system. The principles of the model are derived from a set of key facts concerning cell chemistry, which suggest that cell types, and their associated cell cycles, can be considered as attractors of the dynamics of interacting gene expression levels. The specific form of gene interactions used to achieve this goal is inspired by combining two strands of neural network modelling (i) storage of (limit) cycles and (ii) storage of hierarchically organised attractors.
This paper is intended to provide a proof of concept of this type of modelling approach. We thus decided to in-vestigate the simplest possible hierarchy of cell types that allows us to test our approach, viz. a two-level hierarchy consisting of the stem cell and a single layer of differen-tiated cells derived from it. Furthermore, we chose to consider only interactions between pairs of binary gene expression levels.
In the present study we show that cell reprogramming is possible using either an undirected approach, which consists of increasing the noise level in the dynamics, or an approach that relies on direct perturbations between specific phases of the cell cycle. Two key non-trivial results appear from our model. Firstly, it takes multiple generations of the cell cycle for a progenitor to be repro-grammed to a stem cell, as it transitions through inter-mediate states which show similarity with both the initial and final state. Also, a finite fraction of gene expression levels need to be perturbed in order to reprogramme a cell.
We assume that there are states in the cell cycle were mutual similarity in gene expression levels between dif-ferent cell types is large. These stages of the cell cycle are then natural targets for perturbations to induce changes in cell type. The fraction of genes that need to be perturbed in order to reprogramme a cell depends on the stage of the cell cycle to which the perturbations are applied, as well as the noise level of the system. At low noise levels, this number was much larger than that required in the Yamanaka reprogramming experiments, though we found it to decrease substantially with increasing noise levels. The “true” noise level of a cell is difficult to quantify, but our model allows for reprogramming in both low and high noise regimes.
As far as the authors are aware, gene expression levels in different levels of the cell potency hierarchy or in different phases of the cell cycle are still not well characterized. In the present study we have used a scenario where gene expression levels in differentiated cells are slightly lower than in a stem cell during the same state of the cell cycle, and we have taken one of the cycle phases to have lower levels of gene expression than the others (thereby increasing mutual similarities of different cells in this cycle phase). We have checked that de-differentiation along the two different routes, noise induced and via directed perturbations, does not depend on these specific choices, although details, such as critical thresholds for reprogramming, do change as scenarios are modified.
There are some limitations to our modelling approach. Firstly, de-differentiation is modelled in terms of the entire genome of a cell, whereas only a subset of gene expression levels could be responsible for cell fates and differentiation. Our model can be adapted to consider only a sub-network of gene expression levels which are responsible for cell fates. In order to maintain the large diversity of cell types, higher order interactions will be required to stabilise and store a large number of attractors M. This is biologically reasonable, since proteins expressed from multiple genes can form complexes, which are transcription factors, and genes can often require proteins binding to promoter sites and enhancer regions before the gene is expressed. Using discrete time dynamics excludes the possibility of variability in gene expression levels in a given cell cycle phase. Therefore any in-cycle dynamics is missed, such as the cell signalling cascades. Finally, we only use rough estimates for the average gene expression levels in numerical experiments and simulations.
One possibility for extending our model would be to relax the choice of independent cell cycle states. Correlated cell cycle phases can be incorporated into the two level hierarchy by changing the way in which the cell cycles are constructed in the model. One could achieve this using a three level hierarchy to store all cell cycles, whilst maintaining the feature that all descendants are a single differentiation from the stem cell cycle. In this situation the root of the hierarchy would be a template of the stem cell expression levels, the second level would then be constructed from this and represent each stage of the stem cell cycle. The newly included third level would consist of each daughter cell type branching off from the corresponding stem cell cycle phase. Such a set up would the be analogous to a two level hierarchy for each cell cycle phase with correlations in the gene expression levels along the cell cycles of each cell type.
In the future the authors hope to work on applying this model to real data in order to test its validity and aid in the design of reprogramming protocols. However, since there is still much to be learnt about cell reprogramming and the decisions of cell fates in developmental biology, the model has been presented here to encourage the discussion between experimentalists and those from a more theoretical background (such as statistical physics, the field which inspires much of this work), because clearly a mixture of these skills will be required in driving our understanding of cell fates and reprogramming forward.
Competing interests:
We declare we have no competing interests.
Funding
R.H. is supported by the EPSRC Centre for Doctoral Training in Cross-Disciplinary Approaches to Non-Equilibrium Systems (CANES, EP/L015854/1).
Authors’ contributions
All authors conceived the model and designed the experiments. R.H. performed the experiments and analysed the data. All authors wrote the paper.
Appendix A: Differentiation transition matrices
To derive the daughter cell expression levels from the stem cells a transition matrix, W, was used. When W is applied to the probability distribution of the a stem cell phase the result corresponds to the distribution of the daughter state, i.e. W .If we define the probability gene is switched on in the differ-entiation from the stem to daughter cell as γρμ, for the same activities in each cell cycle phase (ap = aρμ), then we can define W as the following matrix.
To have (A1) defined as a transition matrix, its columns must sum to one and each element W(ηρμ|ηρ) ∊ [0,1]. Since aρ and aρμ ∊ [0,1] by definition, we must obey the following constraint on γρμ for itself and (A1) to be defined as probabilities,
Appendix B: Equations of motion
In each time step the state of the system is updated based on the field local fields of each site. That is, the expression level of gene i, at time t, depends on the value of the field at time t − 1, i.e. where Θ(x) is the Heaviside step function (Θ(x) = 0 for x ≤ 0 and Θ(x) = 1 for x > 0), and ξi(t) is thermal noise at the site i (with ).
The expected value of site i can be obtained by averaging (B1),
Then using the definitions of the dynamics order param-eters (8) and (9), following expressions can be obtained, when the N → ∞ limit is taken, by making use of the law of large numbers: where 〈…〉ηρ,ηρμ is shorthand for an average over the statistics of (correlated) stem and daughter cell expression levels throughout their cycles,
A more rigorous calculation to determine these equations of motion can be done by following the reasoning in Coolen et al28.
Appendix C: Order parameters for specific cell cycle configurations
This appendix contains the calculation of the order parameters, and , when the system is in different generations of the cell hierarchy. First, when the system is in the daughter cell cycle configuration. Equation (8) can be rewritten as follows by making use of the law of large numbers. (N → ∞) with ,
If ρ̅ ≠ ρthen the daughter cell cycle phase is independent of the stem cell cycle phase and the expectation value factorises, and since 〈ηρ – aρ = 0
However, if ρ̅ = ρ then (C1) can be written as, where 𝔼[x] and 𝔼[x|y] represents the expectation value of x and of x given y, respectively. Thus, provided aρμ > 0 and γρμ < 1, there is a non-zero value for the order parameter. This can be rewritten by noticing that the the numerator is the covariance between ηρ and ηρμ and the denominator is the variance of ηρ. Thus, for ∀i,
Next, when the system is in the stem cell cycle configuration. Following similar reasoning to the above, (9) can be written as, where again it is trivial that mρμ= 0 due to indepen-dence if ρ̅ = ρ.
For the case ρ̅ = ρ,
Then since = 1 and by their definitions even if ρ̅=ρ. Thus, any phase of the stem cell cycle has zero overlap with any phases of all daughter cycles.
Appendix D: Normalised Hamming distance
The normalised Hamming distance between two vectors x = (x1, x2…xN) and y = (y1, y2…yN) is defined as follows,
In the large N limit we can replace the sum over N using the law of large numbers to obtain,
Thus, for the same phase of the cell cycle the Hamming distance between the stem and daughter cell cycles is given by, where the averages were preformed over the conditional the joint probability distribution P(ηρ,ηρμ) = 𝕎(ηρμ|ηρ)P(ηρ). Similarly the Hamming distance between a daughter cell cycle phase and the next phase of the stem cell cycle is given by, where the average was preformed over the joint probability, P(ηρ+1,ηρμ) = P(ηρ+1)P(ηρμ) where the factorisation is due to the independence between the different cell cycle phases.
Acknowledgments
The authors would like to thank Attila Csikász-Nagy and Ignacio Sancho-Martinez for the insightful discussions during the formulation of the model.
Footnotes
↵a Electronic mail: ryan.hannam{at}kcl.ac.uk