Abstract
Inside individual cells, protein population counts are subject to molecular noise due to low copy numbers and the inherent probabilistic nature of biochemical processes. Such random fluctuations in the level of a protein critically impact functioning of intracellular biological networks, and not surprisingly, cells encode diverse regulatory mechanisms to buffer noise. We investigate the effectiveness of proportional and derivative-based feedback controllers to suppress protein count fluctuations originating from two noise sources: bursty expression of the protein, and external disturbance in protein synthesis. Designs of biochemical reactions that function as proportional and derivative controllers are discussed, and the corresponding closed-loop system is analyzed for stochastic controller realizations. Our results show that proportional controllers are effective in buffering protein copy number fluctuations from both noise sources, but this noise suppression comes at the cost of reduced static sensitivity of the output to the input signal. Next, we discuss the design of a coupled feedforward-feedback biochemical circuit that approximately functions as a derivate controller. Analysis using both analytical methods and Monte Carlo simulations reveals that this derivative controller effectively buffers output fluctuations from bursty stochastic expression, while maintaining the static input-output sensitivity of the open-loop system. As expected, the derivative controller performs poorly in terms of rejecting external disturbances. In summary, this study provides a systematic stochastic analysis of biochemical controllers, and paves the way for their synthetic design and implementation to minimize deleterious fluctuations in gene product levels.
I. INTRODUCTION
Advances in single-cell technologies over the last decade have revealed striking differences between individual cells of the same population. For example, the level of a given protein can vary considerably across cells within a population, in spite of the fact that cells are identical clones of each other and are exposed to the same environment [1]–[6]. Such intercellular stochastic differences in gene expression patterns have tremendous consequences for biology and medicine [7]–[12], including stochastic cell-fate assignment [13]–[17], microbial bet hedging [18], [19], bacterial and cancer drug-resistance [20], [21].
Stochastic variations in the level of protein primarily arise from two main sources:
Low-copy number fluctuations in underlying biomolecular components (genes, mRNA, proteins). Moreover, this shot noise is amplified by the fact that transcription of genes is not a continuous process but happens in sporadic bursts [22]–[26].
External disturbances in the protein synthesis rate due to fluctuations in expression-related machinery (RNA polymerases, Ribosomes, etc.) or intercellular differences in cell-cycle stage/cell size [27]–[30].
Given these noise sources, cells encode diverse regulatory mechanisms to suppress stochasticity in the level of a protein around a set point. Perhaps the simplest example of this is a negative feedback loop, where the protein directly or indirectly inhibits its own synthesis [31]–[43]. Such naturally-occurring feedbacks have been shown to be key motifs in gene regulatory networks [44]. Furthermore, design of in-vitro/in-silico synthetic feedback system based on linear PID or nonlinear controllers is an intense area of current research [45]–[53]. In this contribution, we investigate design of biochemical circuits that function as approximate proportional and derivative-based controllers, and systematically investigate their effectiveness in buffering protein noise levels.
In Section II, we introduce an open-loop model of stochastic gene expression where the protein is expressed in random bursts, and its expression is impacted by an upstream noisy input (Fig. 1). We provide exact analytical formulas for the protein mean and noise levels in open loop. Section II also introduces the mathematical tools to be used throughout the paper for the analysis of stochastic dynamical systems. In Section III and IV, we discuss designs of nonlinear biochemical circuits that function as approximate proportional and derivate controllers, respectively. Given the nonlinearities introduced by feedback loops, we use the linear noise approximation method [54], [55] to investigate their noise suppression properties, and validate the results by performing exact stochastic simulations of the feedback system.
Symbols and Notation
Throughout the paper we denote chemical species by capital letters, and use corresponding small letters for molecular counts. For example, if Y denotes a protein species, then y(t) is the number of molecules of Y at time t inside the cell. We use angular brackets to denote the expected value of random variables and stochastic processes. Given a scalar random process y(t) ∈ {0, 1, 2, …} that takes non-negative integer values, then represent its mth order uncentered moment and 𝕡(y(t) = i) is the probability of having i molecules. Steady-state statistical moments are denoted by
Finally, noise in the level of protein species is quantified by the steady-state coefficient of variation squared (variance divided by mean squared) that is defined as
II. SYSTEMS MODELING OF GENE EXPRESSION
We start by introducing simple models of the gene expression process with a particular focus on incorporating noise mechanisms that drive fluctuations in the level of a protein.
A. Incorporating bursty dynamics
Transcription of individual genes inside single cells has been shown to occur in bursts of activity, followed by periods of silence [56]–[61]. Each burst corresponds to the gene stochastically switching to a transcriptionally active state, and then becoming inactive after synthesizing a few messenger RNA (mRNA) transcripts. These mRNAs are typically unstable with short half-lives, and each mRNA decays after translating a few protein molecules. The combined multiplicative effect of both these processes (single gene making multiple mRNAs, single mRNA making multiple proteins) is to create a net burst of protein molecules, every time the gene becomes active. Motivated by these experimental findings, we phenomenologically model protein copy-number fluctuations via a bursty birth-death process [62]–[68]. More specifically, bursts arrive at a constant Poisson rate ky that corresponds to the frequency with which the gene becomes active. Each bursts arrival event, results in the synthesis of By∈{1, 2, …} protein molecules, where the burst size By is an independent and identically distributed random variable that is drawn from an arbitrary positively-valued probability distribution.
Let y(t) denote the intracellular copy number of protein Y at time t. Then, based on the above model description, the probability of a burst event of size By = j molecules occurring in the next infinitesimal time interval (t, t + dt] is
Assuming each protein molecule decays with a constant rate γy, defines the probability for the protein death event occurring in the time interval (t, t + dt] as
Having defined an integer-valued continuous-time Markov process y(t) via the probabilities (4)-(5), we now focus our attention on its statistical moments. We refer the reader to [69]–[72] for a thorough analysis of moment dynamics for stochastic systems of the form (4)-(5), and only provide the main result here – the time evolution of the expected value of y(t)m is given by where the infinitesimal generator G takes the form
Intuitively, the right-hand-side of (7) is simply the product of the change in ym when an event occurs and the probabilistic rate at which it occurs, summed across all possible events. Substituting the appropriate value of m in (6) yields the following moment dynamics where ⟨By⟩ is the mean protein burst size, and is its second-order moment. Subsequent steady-analysis of (8) reveals the protein mean and noise levels to be respectively. By = 1 with probability one leads to Poissonian fluctuations in Y copy numbers with . If the burst size By is assumed to be a geometrically-distributed random variable with mean burst size ⟨By⟩ (as shown experimentally for an E. coli gene [73]), then , and the above noise levels reduce to
A key point worth mentioning is that the product is independent of the burst frequency ky, while in (10) is independent of the mean burst size ⟨By⟩. Thus, simultaneous measurements of both the mean and protein noise levels allows for discerning whether a change in is a result of alterations in ky or ⟨By⟩. Interestingly, this noise-based method works quite well in practice, and has successfully elucidated the bursty kinetics of several genes [74]–[77].
B. Incorporating external disturbance
Next, we introduce another important source of stochasticity that arises from external disturbances in the protein synthesis rate. These disturbances correspond to fluctuations in the abundance of enzymes, such as, transcription factors, RNA polymerases, etc. We lump these factors into a single species X and model its stochastic dynamics via a bursty birth-death process analogous to (4)-(5):
Here kx is the arrival rate of bursts in X, Bx is the burst size, and γx is the decay rate of X. Then, as per (9)
The disturbance is connected to the synthesis of Y by assuming that the frequency of protein Y bursts is proportional to x(t), and is given by . The division by ensures that the average burst arrival rate is ky. This leads to a system of coupled bursty birth-death processes given by (11) and
The statistical moments of this joint process evolve as per
[69]–[72]. To write moment dynamics in a compact form we define a vector that consists of all the first and second order moments of x(t) and y(t). Then, its time evolution is given by a system of linear differential equations where vector â and matrix A are obtained via (15) by choosing appropriate values of m1, m2. Steady-state analysis of (16) results in the same mean Y level as (9), and the following noise level that can be decomposed into two components. The first component is the noise contribution from stochastic bursts computed earlier in (9), and has been referred to in literature as the intrinsic noise in Y [78]–[82]. The second component is the noise contribution of the external disturbance, and has been referred to as the extrinsic noise in Y. Note that the ratio γy/(γy + γx) quantifies the time-averaging of upstream fluctuation in X by Y. For example, fast fluctuations in X are efficiently averaged out by Y, and this ratio approaches zero for γx→∞. In contrast, slow fluctuations in X lead to inefficient time-averaging that increases Y noise levels to
Next, we investigate how negative feedback regulation suppresses different noise components in (17) to minimize fluctuations in Y copy numbers around it mean .
III. NOISE SUPPRESSION USING PROPORTIONAL CONTROLLER
To implement a negative feedback loop we first introduce a new protein species Z that functions as a noisy sensor of Y. Protein Z is also assumed to be synthesized in bursts of size Bz, and senses Y via its burst frequency kzy(t) that responds linearly to any changes in Y levels. This leads to the following bursty birth-death process for z(t) where γz is the decay rate of protein Z. Recall from Section II-B that the frequency of bursts in the Y protein was in the open-loop system. To close the feedback loop, we now modify this burst frequency to , where g(z) is a positively-valued monotonically decreasing function of z(t). Typically, g takes the form of a Hill function that mechanistically arises from the fast binding-unbinding of the protein to the gene’s promoter region to regulate transcriptional activity [83]. Within this feedback there are three noise mechanisms at play: external disturbance X impacting synthesis of Y, expression of Y in stochastic bursts, and a noisy sensor Z that measures Y and inhibits it (Fig. 2). The overall stochastic system is given by (11), (19) and
A. Analysis of Mean levels
At equilibrium, the mean levels of the random processes x(t), y(t) and z(t) satisfy
Assuming copy-number fluctuations are tightly regulated by the feedback system, and that they are small,
Given that g(z) is a positively-valued monotonically decreasing function, using (21) and (22), the steady-state mean level of Y is the unique solution to the equation
Having solved for the means, the burst frequency of Y can now be approximated using Taylor series as Where is the log sensitivity of the function g evaluated at steady state. Note that if the sensor dynamic is very fast compared to the measurand Y (i.e., γz » γy), then z(t) ∝ y(t), and the burst frequency in (24) will be proportional to the error . Hence, this circuit architecture can be interpreted as an approximate proportional controller with feedback gain fp. Finally, if we consider the parameter ky in Y ‘s burst frequency as an environmental input, then one can define a static sensitivity of to ky which using (21) and (25) is given by and monotonically decreases with increasing gain. Note for the open-loop system fp = 0 and as mean is simply proportional to ky from (9).
B. Analysis of Noise levels
Next, we focus on computing the noise levels in Y for the overall feedback system. As before, we define a vector μ that consists of all the first and second order moments of x(t), y(t) and z(t). The time evolution of μ can be obtained by expanding (15) to the three-species system, where Y ‘s nonlinear burst frequency is replaced by its linear approximation (24). Having linear probabilistic rates for all birth-death events results in a linear dynamical system (16) that can be solved analytically to obtain steady-state moments [72]. This analysis yields the following noise level for protein Y which can be decomposed into three components. The first component is the intrinsic noise in Y due to its bursty expression, and it decreases with increasing feedback gain fp approaching a non-zero lower bound as fp →∞. This lower bound represents a fundamental limit to which intrinsic noise can be decreased, and this limit is determined by how fast the sensor dynamics is compared to Y ‘s decay rate. The second component is the noise contribution from the external disturbance that monotonically decreases to zero as fp →∞. The third component arises from the fact that the sensor Z is itself noisy, where is the noise in Z due to its own expression occurring in random bursts. This third component is amplified with increasing feedback gain, and as a consequence, the total noise is a non-monotonic function of fp with noise being minimal at an optimal feedback strength (Fig. 2). When fp = 0, (28) reduces to the open-loop noise (17).
To further simplify the formula we assume that sensor dynamics is sufficiently fast (γz » γy), and the time-scale of disturbance fluctuations are slow (γx « γy). In this case, the sensor noise contribution becomes minimal, and (28) simplifies to
Note that the contribution from external disturbance decreases as compared to 1/ fp for the intrinsic noise. Hence, proportional feedback is much more effective in buffering stochasticity from external inputs rather than the intrinsic noise. This point relates to the static sensitivity defined in (26), where increasing feedback gain suppresses noise, but it comes at the loss of adapting Y levels to changes in the environmental input.
IV. NOISE SUPPRESSION USING DERIVATIVE CONTROLLER
Having completed the analysis for a proportional controller we next turn our attention to a derivative controller.
A. Derivative controller design
How can biochemical circuits be designed to approximately sense the derivative of y(t)? To see this, consider the sensor dynamics in the deterministic limit which in the Laplace domain corresponds to
Here Z(s) and Y (s) are the Laplace transforms of z(t) and y(t), respectively, and we have used the fact that at equilibrium . Now consider the scenario where Z activates the burst frequency of Y, and Y inhibits it own burst frequency (Fig. 3). This created an incoherent feedforward circuit that feedbacks into Y. Let the burst frequency of Y be proportional to (z/y)h, which corresponds to both Y and Z regulating the burst frequency with the same Hill coefficient h, and a strong binding affinity of the repressor Y [83], [84]. Then, in the limit of small fluctuations in z(t), y(t) around , respectively,
Using (32) and assuming γz is large (i.e., fast sensor dynamics), the Laplace transform of the right-hand-side of (24) is
Recall that sY (s) is the Laplace transform of the time derivate of y(t), and hence in the time-domain, the burst frequency (24) corresponds to implementing a derivative controller. Going back to the original stochastic system, let the frequency of bursts in the Y protein be given by . Then, as the ratio becomes a constant at steady-state, the mean protein level for Y is proportional to ky and the sensitivity as in the open-loop system.
B. Analysis of noise levels
To perform a noise analysis of the derivative controller-based feedback system, we revert to the noisy sensor Z described by the bursty birth-death process (19). The stochastic dynamics of protein Y is now described by
As before, the external disturbance is described by (11). To write a closed systems of differential equations for the time evolution of moments, we linearize protein Y ‘s burst frequency assuming small copy-number fluctuations
Where is the derivative feedback gain. A steady-state analysis of the resulting linear moment dynamics yields the following noise in protein Y
Analysis of the resulting noise components reveals that both the intrinsic noise, and the noise contribution from the external disturbance, decrease with increasing gain fd, with the former showing a much faster decay (Fig. 3). The noise contribution from the sensor amplifies with increasing feedback gain resulting in the total noise being minimized at an intermediate gain (Fig. 3). In the limit of slow fluctuations in the external disturbance γx « γy, γz, the above noise level simplifies to showing the derivative controller’s inability to reject low-frequency external disturbances. Finally, assuming no external disturbance , we verify the ability of a derivative controller to minimize intrinsic noise in Y by performing exact Monte Carlo simulations based on the Stochastic Simulation Algorithm (SSA) [85]. Stochastic simulation results of the overall nonlinear feedback system are shown in Fig. 4, and the noise levels show a good match with the formula (39) confirming the noise suppression abilities of a derivative controller.
V. CONCLUSION
While PID controllers have become quite standard in industry, designing biochemical circuits that perform analogous functions inside cells is a highly nontrivial problem. Here we present simple circuits that function as approximate proportional and derivative controllers assuming fluctuations in molecular counts are small around their respective means. Our analysis of biochemically-implemented proportional feedback reveals the following properties:
Proportional feedback is more efficient in suppressing stochasticity arising from noisy input signals, compared to noise arising from protein expression occurring in random bursts (Fig. 2).
Any form of measurement noise (for example, due to stochastic expression of the sensor protein), leads to an optimal feedback gain for minimizing total protein noise, reminiscent of traditional feedback controllers.
Noise suppression comes at the cost of reduced static input-output sensitivity, i.e., the protein levels are precisely regulated for a given environment, but do not respond to new environments.
We further provide design of a biochemical circuit for derivate-based control. In essence, the derivative of a signal is sensed by taking the difference of a delayed-signal (the sensor output) and the original signal. Intriguingly, our analysis shows that this controller suppresses intrinsic noise in the protein while preserving the open-loop static input-output sensitivity (Fig. 3). Intuitively, any rapid increase in protein levels due to a random burst is compensated by lowering the frequency of subsequent bursts. We confirmed our small-noise analysis results with exact Monte Carlo simulations of the nonlinear feedback system. As part of future work, we will investigate biochemical networks for integral feedback control, and constructing biological PID controllers for a given static input-output sensitivity, noise in the target protein, and transient response.
ACKNOWLEDGMENT
AS is supported by the National Science Foundation Grant grant ECCS-1711548.
Footnotes
↵1 Departments of Electrical and Computer Engineering, Biomedical Engineering at the University of Delaware, Newark, DE 19716, USA. absingh{at}udel.edu, saurmodi{at}udel.edu, sdey{at}udel.edu
REFERENCES
- [1].↵
- [2].
- [3].
- [4].
- [5].
- [6].↵
- [7].↵
- [8].
- [9].
- [10].
- [11].
- [12].↵
- [13].↵
- [14].
- [15].
- [16].
- [17].↵
- [18].↵
- [19].↵
- [20].↵
- [21].↵
- [22].↵
- [23].
- [24].
- [25].
- [26].↵
- [27].↵
- [28].
- [29].
- [30].↵
- [31].↵
- [32].
- [33].
- [34].
- [35].
- [36].
- [37].
- [38].
- [39].
- [40].
- [41].
- [42].
- [43].↵
- [44].↵
- [45].↵
- [46].
- [47].
- [48].
- [49].
- [50].
- [51].
- [52].
- [53].↵
- [54].↵
- [55].↵
- [56].↵
- [57].
- [58].
- [59].
- [60].
- [61].↵
- [62].↵
- [63].
- [64].
- [65].
- [66].
- [67].
- [68].↵
- [69].↵
- [70].
- [71].
- [72].↵
- [73].↵
- [74].↵
- [75].
- [76].
- [77].↵
- [78].↵
- [79].
- [80].
- [81].
- [82].↵
- [83].↵
- [84].↵
- [85].↵