Abstract
Policing occurs in complex insect, animal and human societies, where it is used as a conditional strategy to prevent cheating and enforce cooperation. Recently, it has been suggested that policing might also be relevant in enforcing cooperation in much simpler organisms such as bacteria. Here, we used individual-based modelling to develop an evolutionary concept for policing in bacteria, and identify the conditions under which it can be adaptive. We modelled interactions between cooperators, producing a beneficial public good, cheaters exploiting the public good without contributing to it, and public-good-producing policers that secrete a toxin to selectively target cheaters. We found that toxin-mediated policing is favored when (i) toxins are potent, (ii) cheap to produce, (iii) cell and public good diffusion is intermediate, and (iv) toxins diffuse farther than the public good. Overall, we show that toxin-mediated policing can enforce cooperation, but the parameter space where it does so is quite narrow. Moreover, we found that policing decays when the genetic linkage between public good and toxin production breaks. This is because policing is itself a public good, offering protection to toxin-resistant mutants that still produce public goods, yet no longer invest in toxins. Our work suggests that very specific environmental conditions are required for genetically fixed policing mechanisms to evolve in bacteria, and offers empirically testable predictions for their evolutionary stability.
Introduction
Cooperation, a process where individuals act to increase the fitness of others at an immediate cost to themselves, is common across the tree of life [1, 2]. While cooperation is widespread in humans [3] and higher organisms such as insects and vertebrates [4, 5], we have only recently recognized that also microbes have evolved adaptive cooperative behavior [6–9]. Types of microbial cooperation include the formation of biofilms, where individuals secrete polymeric compounds to build a protective extracellular matrix [7], the formation of fruiting bodies where some individuals sacrifice themselves to enable the dispersal of others [10]; and the secretion of shareable metabolites, such as proteases to digest nutrients [11, 12], siderophores to scavenge extra-cellular iron [13, 14], and biosurfactants enabling group motility [15].
Although cooperation is thought to provide benefits for the collective as a whole, it is intrinsically vulnerable to exploitation by cheating mutants, which benefit from the cooperative acts performed by others, but refrain from contributing themselves to the welfare of the group [16]. Thus, there has been great interest in identifying mechanisms that maintain cooperation and prevent the invasion of cheaters [2, 17, 18]. Work on microbes have proved particularly useful in this context because cheating mutants can easily be engineered and factors important for cooperation can be experimentally manipulated in laboratory settings. Studies following this approach revealed a plethora of mechanisms promoting cooperation, including: (i) limited dispersal ensuring that cooperators stay together [19, 20]; (ii) molecular mechanisms allowing the recognition of other cooperators [21–23]; (iii) regulatory linkage between multiple traits imposing additional costs on cheaters [24–28]; and (iv) mechanisms reducing the cost of cooperation such as public good recycling [29] or the use of superfluous nutrients for public goods production [30, 31].
In addition, several studies suggested that bacteria have evolved policing mechanisms, which enable cooperators to directly sanction cheaters and thereby enforce cooperation [32–36]. For example, Wang et al. [34] showed that in the opportunistic pathogen Pseudomonas aeruginosa, the synthesis of a publically shareable protease is regulatorily coupled to the synthesis of the toxin cyanide in such a way that cheaters, deficient for protease production, simultaneously lose immunity against cyanide. Thus, the costly synthesis of a harmful toxin, which selectively targets non-cooperative cheaters, can be understood as a policing mechanism. While it seems intriguing that organisms as simple as bacteria can perform policing behavior, multiple questions regarding the evolution of microbial policing have remained unaddressed. For one thing, we know little about the environmental conditions promoting policing via toxin production. In this context, one would expect that the spatial structure of the environment, which affects the diffusion of cells, public goods and toxins, should play a crucial role [37–39]. Moreover, it is unknown how potent a toxin must be and how much it can cost in order to efficiently fight cheaters. Finally, it remains unclear whether policing is an evolutionary stable strategy or whether non-policing cooperators, which are immune against toxins can exploit policers as second order cheaters [33], as it is the case in higher organisms [40].
Here, we address these issues by using realistic individual-based models that simulate interactions between cooperating, policing and cheating bacteria on a toroidal two-dimensional surface [39]. In our in-silico approach, bacteria are modeled as discs and seeded in low numbers onto the surface of their habitat, where they can consume resources, grow, divide, disperse, and secrete compounds according to specified parameters. Important to note is that both bacteria and public good molecules are modelled as individual agents and are free to diffuse on a continuous landscape, closely mimicking natural conditions. We considered four different bacterial strain: (1) a wildtype cooperator strain producing a beneficial public good; (2) a policing strain producing a toxin and an immunity protein in addition to the public good; (3) a cheating strain neither producing the public good, the toxin nor the immunity protein; and (4) a cooperator strain producing the public good and the immunity protein, but not the toxin. We assume tight linkage between all three traits, such that (3) directly evolves from (2), matching the policing mechanism proposed by Wang et al. [34].
In a first set of simulations, we examined the performance of the cooperator, cheater and policing strains in monoculture to understand how the relative costs and benefits of public good and toxin production affect strain fitness. Next, we competed wildtype cooperators against cheaters across a range of bacterial and public good diffusion coefficients to determine the parameter space where cooperation is favored in the absence of policing [39]. Subsequently, we competed the cheater against the policing strain across the same parameter space, to test whether policing extends the range of conditions across which cooperation is favored. Finally, we simulated the situation where toxin-resistant public good producers evolve from the policing strain, and ask whether policing via toxin production itself constitutes an exploitable public good.
Results
The boundary costs of policing vary in response to environmental diffusivity
To assess how costly policing can be in order to still generate a net benefit of cooperation, we compared the growth of the policing strain (P) in monoculture, to the growth of the wildtype non-policing cooperator (W) and the strain producing neither public goods nor policing toxins (C). As expected, C was not affected by the diffusivity of the environment, but grew at a constant rate determined by its basic growth rate μ, reaching carrying capacity after 12,000 time steps (Fig 2). Conversely, the growth of W was reduced in more diffusive environments, where the likelihood of public good sharing and consumption declines. Nonetheless, W grew significantly better than C under all conditions, demonstrating the benefit of public goods cooperation (Fig 2). The performance of P greatly varied both in response to the relative costs of policing (cpg/ctox) and the diffusivity of the environment. In environments characterized by low diffusion, P always outperformed C even when the cost of toxin production was four times higher than the cost of public good production (Fig 2A). This was no longer the case in environments characterized by intermediate or high diffusivity, where P only grew better than C when toxins were cheaper to produce than public goods (Fig 2B and 2C). If this condition was not met, then the high costs of policing combined with reduced public good consumption decreased cell growth to a point that prevented populations from reaching carrying capacity. Based on these results, we chose cpg/ctox = 1.5 for all subsequent simulations.
Interestingly, the relationship between the cost ratio r = cpg/ctox and the time to reach carrying capacity (τK) was best captured by the Monod equation [41], a hyperbolic equation initially used to explain the exponential growth rate as a function of nutrient concentration.
Applied to our system, we found that the function provides a fair approximation to relate r to τK where Rk represents the ratio limit for reaching carrying capacity and Rτ the time at which the carrying capacity is half the maximum (Fig S1).
However, the equation does not fully capture the observed pattern when the ratio approaches one, namely when ctox ≈ cpg, under conditions of low environmental diffusivity. This is partially understandable since R(τK) does not directly take the viscosity of the medium into account.
In the absence of policing, cooperation is favored when cell diffusion is low
When competing W against C without a policing mechanism in place, cooperation was only strongly favored when cells did not diffuse (Fig 3A). In other words, if cooperator cells grew as microcolonies, physically separated from the cheaters, then efficient public good sharing among cooperators is promoted (see [19, 42, 43] for experimental support). We further found that cooperators and cheaters could coexist when cell and public good diffusion was low, but greater than zero (Fig 3A). Under all other conditions, cheaters strongly dominated and pushed cooperators to very low frequencies. These results are in line with our previous simulations, showing that there is only a narrow parameter space within which public good cooperation can be favored [39].
Policing through a potent toxin extends the parameter space favoring cooperation
The introduction of a policing mechanism, which operates via the secretion of a toxin that selectively targets cheaters, had multiple dramatic effects on the competitive outcome between the cheater C and the cooperating policer P (Fig 3B). With a potent toxin (θT = 1000), policing strongly increased selection for cooperation under conditions where cooperators could previously only coexist with cheaters (compare Fig 3A and 3B for combinations of low public good and cell diffusion). This demonstrates that policing can indeed extend the parameter space across which cooperation can be favored. Conversely, we found that policing also had negative effects and drastically accelerated selection against cooperation (Fig 3B), especially under conditions where cheaters previously experienced only a moderate selective advantage (compare Fig 3A and 3B for combinations of intermediate public good and cell diffusion). These two opposing effects led to a sharp transition between conditions that either completely favor P or C, leaving very few conditions where coexistence between the two strains is possible.
Toxin diffusion and potency affect selection for cooperation
We found that toxin diffusion significantly affected the parameter space across which cooperation was favored (Fig 3B and 3C). While we manipulated toxin diffusion across a gradient from 1 to 7 μm2/s in steps of 1, we here only present the two extremes (Fig3B with dt = 1, and Fig 3C with dt = 7), as the outcomes of the other treatments resulted in intermediate patterns. Our simulations revealed that increased toxin diffusion was favorable for cooperation, especially under conditions of relatively high public good diffusion but low cell diffusion. These results show that policing is particularly efficient when toxins are sent away to target more distant competitors whilst keeping the public good more locally for sharing among clonemates.
Our simulations further revealed that high toxin potency is crucial for policing to promote cooperation (Fig 3D and 3E). While this finding seems trivial at first sight, the dramatic effects we observed when decreasing toxin potency are remarkable. In our simulations, we manipulated toxin potency by varying θT, the number of toxin molecules sensitive cells can tolerate before they die (θT varied from 1000 to 2200 in steps of 400). Important to note is that larger values of θT did not only increase toxin tolerance, but also the latency phase during which toxins had little effects on growth (see equation 3). Given this context, we found that a reduction of toxin potency by 45 % (Fig 3D, θT = 1000 versus Fig 3E, θT = 1800) already negated any benefit of policing, and even increased the parameter space across which cooperation was selected against.
Policing is not an evolutionary stable strategy if genetic linkage between traits breaks
In a next set of simulations, we asked whether policing is an evolutionary stable strategy or whether it can be invaded by a non-policing strain that produces the public good, but is resistant against the policing toxin (R). Such mutant strains can arise when the genetic linkage between the three traits (public good, toxin, immunity) is broken. We simulated this situation by competing the policing strain P against R in the presence of the cheater C in nine environments differing in their diffusivity (Fig 4). Here, we focused on a subset of combinations of diffusion coefficients (Dc= 1, 1.5, 2; dp= 3, 4, 5; and dt= 7 μm2/s). We chose this parameter space because it comprises all possible outcomes from the competitions between P and C (Fig 3C: five cases of P dominance, two cases of C dominance, and two cases of coexistence).
We found that the addition of R to the system consistently drove P to extinction regardless of the diffusivity of the environment (Fig 4). Our simulations, which kept populations at half the carrying capacity (K/2 ~500 cells) for an extended period of time (roughly 10 times longer than in the previous pairwise competition assays), allowed us to distinguish three distinct competition phases. The first phase comprises the time frame in which the community grows from three cells to K/2. During this phase, we observed cyclical fluctuations of strain frequencies with a general tendency for C to increase, P to decrease, and R to remain stable. The cyclical patterns observed here are reminiscent of the rock-paper-scissors dynamics described by previous studies on bacterial interactions [44–46], where strains chasing each other in non-transitive interaction patterns with no overall winner. The second phase was characterized by a pronounced dip in C frequency accompanied by a strong increase in R frequency, and a moderate increase in P frequency in eight of the nine diffusion conditions (Fig 4). This pattern is most likely explained by the accumulation of toxins in the environment, which efficiently suppressed C, but at the same time gave R leverage, as it could benefit from the effect of policing without paying the cost for it. During the third phase, we observed the extinction of P, the concomitant recovery of C followed by a decrease of R (Fig 4). These patterns arise because as P decreases, toxin concentration declines allowing the recovery of C, which then efficiently exploit R. In all cases, our simulations returned to a simple cooperator-cheater scenario, with the relative success of the two strategies being determined by the diffusivity of the environment (as shown in Fig 3A).
Discussion
Understanding how policing can repress competition and foster cooperation in social groups of higher organisms has attracted the attention of evolutionary biologists for decades [2, 47–53]. Here, we developed a conceptual evolutionary framework to examine whether policing could also be an effective way to enforce cooperation in groups of microbes, sharing beneficial public goods. In our models, policing is exerted via the secretion of a toxin that specifically targets cheater cells, which do not contribute to the pool of public goods. Our simulations reveal that policing is most conducive under conditions of intermediate cell dispersal and public good diffusion, where it can extend the parameter space, under which cooperation is favored. We further found that an effective policing mechanism entails a toxin that is: (i) cheaper to produce than the public good; (ii) very potent in killing; and (iii) more diffusible than the public good in order to effectively reach cheaters. However, our simulations also reveal two major downsides of policing. First, policing can accelerate the loss of cooperation under conditions where cheaters and cooperators can coexist in the absence of policing. This leads to a sharp state transition between conditions where policing either favors or disfavors cooperation. Second, policing is not an evolutionary stable strategy if the genetic linkage between the production of the public good, and the toxin-anti-toxin complex can be broken. If this occurs then toxin-resistant mutants that produce public goods, but no longer contribute to toxin production, derail policing. This demonstrates that microbial policing itself constitutes an exploitable public good [33].
Microbiologists have recently proposed a number of potential toxin-based policing mechanisms that seem to keep cheaters in check [33–36]. While the idea of bacteria being able to punish cheating community members is exciting, our simulations reveal that the ecological spectrum under which policing can be favored is actually quite narrow. For one thing, we found that policing is not required when environmental diffusivity is low, conditions that promote cooperation per se. The reason for this is that low public good and bacteria diffusion lead to significant spatial structuring of the bacterial community, separating cooperators from cheaters, which promotes the local sharing of public goods among cooperators [19, 20, 43,54]. Moreover, we show that policing is not favorable when environmental diffusivity is high, conditions that break any spatial association between cooperators and their public goods. Toxin-mediated policing is detrimental here because: (i) cheaters can freely exploit public goods; (ii) many toxins get lost due to high diffusion, and thus never reach their target; (iii) and the high level of cell mixing reduces the efficiency and selectivity of policing, as cooperators are hit by toxins as often as cheaters [38]. Overall, it turns out that only intermediate levels of environmental diffusivity proved beneficial for policing. The issue with this narrow parameter space is that environmental diffusivity is likely to vary both temporally and spatially under natural conditions, which could quickly shift the selective balance for or against policing. It thus remains to be seen whether policing can indeed evolve under fluctuating environmental conditions.
Another important point that has received little attention so far concerns the question whether the reported policing mechanisms have indeed evolved for this very purpose or whether they represent by-products of regulatory linkage of traits for other reasons than policing [55]. For instance, in the case of P. aeruginosa it is well conceivable that cyanide primarily serves as a broad-spectrum toxin to target inter-specific competitors under high cell density [56]. This might be the reason why the expression of cyanide is controlled by quorum sensing, and why it is regulatorily linked to other public good traits, such as protease production, whose expression is also controlled in a density-dependent manner [57]. Consequently, the observed cyanide-mediated policing exerted by wildtype strains on protease-deficient strains could be a mere by-product of this regulatory linkage. Alternatively, it is also possible that an initial co-incidental regulatory linkage between cyanide and protease later proved useful as a policing mechanism and evolved as such through cooption [58]. Clearly, further research is needed to uncover the evolutionary history of these putative policing mechanisms, and care must be taken to distinguish between mechanistic (proximate) and evolutionary (ultimate) explanation of observed behavioral patterns [55].
The potential policing mechanisms reported for microbes and the one implemented in our simulations differ in one important aspect from the policing systems found in higher organisms. Specifically, the difference is that the microbial policing mechanisms are genetically fixed (i.e. strains are either cooperating policers or cheaters), whereas in higher organisms policing is perceived as a conditional strategy, which can be applied to sanction cheaters only if required [2, 47–52, 59]. In the latter scenario, individuals can take decisions on whether to cheat or to cooperate, and whether to impose sanctions or not. In certain cases, it was found that the mere threat of policing was sufficient to coerce individuals to cooperate and prevent cheating in the first place [51, 53, 60]. Microbes clearly lack cognitive abilities for such conditional behaviors, and it is thus not surprising that cheating, cooperating and policing strategies are genetically fixed in these organisms. Nonetheless, we argue that it seems fair to use the term ‘policing’ for the reported behaviors, but also to keep in mind the difference between conditional and fixed strategies.
In summary, our work contributes to the development of an evolutionary concept for policing in microbial systems. It shows how ecological factors, in particular the diffusivity of the environment, interact with the properties of a toxin-mediated policing system, to define the parameter space in which policing can be favored. It further demonstrates how realistic individual-based modelling, tracking both cells and their public goods over time and across space, can be used to deepen our understanding of social interactions in microbes.
Materials and methods
The in-silico landscape and its digital bacterial inhabitants
The in-silico habitat consists of a two-dimensional continuous toroidal surface, with no boundaries. The size of the surface is 60×60 μm2 = 3,600 μm2. Bacteria are modeled as discs with an initial radius of 0.5 μm. Bacteria can consume resources, which are unlimited in our system, grow at a basic growth rate, and divide when reaching the threshold radius of 1 μm. Bacteria can disperse on the landscape according to a specific cell diffusion coefficient D (μm2/s), and are not bound to a grid, but can freely move on the continuous landscape (i.e. we used an off-lattice model with double-precision floating-point format). At the beginning of a simulation, we randomly seed one founder bacteria of each strain onto the surface, free to grow and divide according to its life cycle.
In addition to the basic growth rate μ, the growth of a bacterium is influenced by its social behavior and its interaction with other community members. Costs, reducing growth, incur to individuals producing public goods, toxins and anti-toxins. Additional costs incur to sensitive cells taking up toxins. The uptake of a public good, meanwhile, has a positive effect on growth for the beneficiary. Accordingly, the growth of a wildtype public good producing strain (W), a policing strain producing a toxin together with an anti-toxin in addition to the public good (P), a non-public good producing toxin-sensitive cheating strain (C), and a toxin-resistant public good producing strain (R) are defined by the following set of functions: where μ represents the basic growth rate. The value of μ was set to one for simplicity, and the increase in the cell radius was adjusted such that cell division occurs every 1,200 time steps, when neither public goods nor toxins are produced. The parameters cpg, ctox, cres are the cost of public goods, toxin-anti-toxin, and anti-toxin production, respectively, whereby we assume that the toxin and the anti-toxin are equally costly (i.e. cres = ctox/2). While we set cpg = 0. 001 for each public good molecule produced, we varied ctox in relation to cpg to estimate the acceptable boundary costs of the policing mechanism. The term ∑Pj stands for the number of public goods consumed by an individual and b for the benefit derived from this action. We set b = 0.01 per consumed public good molecule. Finally, the term ∑Ti represents the number of consumed toxins. Toxins decrease the overall growth rate and lead to death when they accumulate beyond the threshold value θT. The negative effect toxins have on growth is further defined by the parameter κ, which we set to 3.5, resulting in a latency function, where the negative effects of toxins on growth accelerate with increased toxin uptake.
The public good producing strains constitutively secrete one molecule per time step, whereas the policing strain additionally secretes toxins at the same rate. The diffusion of the cells and molecules (described by the diffusion coefficients Dc, dp and dt, respectively) were modeled according to a Gaussian random walk, with a Gaussian random number generator based on the Box-Muller transform that converts uniformly distributed random numbers to normally distributed random numbers. Following diffusion, public goods and toxins were consumed whenever there was co-localization of molecules and cells. Both, public goods and toxins were represented by a point on the landscape up to the precision of the computer (double precision). The molecules remain in the simulation until they were either consumed or decayed. The probability of decaying was determined by a durability value δ (set to 500 time steps) and an exponential decay function where ΔT is the current lifespan of a molecule and ω = 0.1 the steepness of the decay.
On our off-lattice landscape, individuals can physically overlap following cell growth and diffusion. To remove the overlaps, we implemented a procedure where cells are moved slightly apart from each other by a random factor scaled by a maximum pulling distance. In a final step, our stimulation involves a life-dead-control of each individual followed by the removal of dead cells. Cells can die because they are sensitive to toxins and have passed the threshold value θT for toxin uptake, or experience negative growth (i.e. they shrink) and fall below the minimal cell diameter of 0. Fig 1 depicts the order in which all the actions in our simulation were executed per time step. This simulation cycle continues until the total number of cells reaches the carrying capacity K = 1000 cells. For each of the simulated parameter settings, we performed 50 independent replicates. During the simulations, we monitored the relative frequency of the individual strains. At the end of a simulation, we recorded the final strain frequency, calculated the mean time between two cell divisions, and the per capita public good and toxin uptake for each individual strain.
Simulating strain performance in monoculture
Policing in the form of toxin production generates additional costs for cooperators. We thus asked how large can ctox be in relation to cpg, in order to maintain a net benefit of cooperation. To address this question, we simulated the growth of W, P, and C in monoculture, and varied cpg/ctox from 4 to 0.25 in seven steps. A net benefit of cooperation is given when the policing strain P grows better than strain C, which neither produces public goods nor toxins. Because we know that the diffusion of cells and public goods influence the efficiency of cooperation [39], we repeated the simulations for three different diffusion regimes, including high (cell diffusion = 4 μm2/s, molecule diffusion = 7.0 μm2/s), medium (cell diffusion = 2.0 μm2/s, molecule diffusion = 3.5 μm2/s) and low (cell diffusion = 0.1 μm2/s, molecule diffusion = 1.0 μm2/s) diffusion. We then compared the time needed by the three strains to reach carrying capacity.
Simulating pairwise strain competitions
Next, we simulated competitions between the strains W and C, and the strains P and C. In both scenarios, C acts as a cheater and exploits the public good produced by either W or P. However, C is harmed by toxins in competition against P, but not when competing against W. While the competition between W and C provides information on the environmental conditions required for cooperation to be favored in the absence of policing, the competition between P and C addresses the question whether the range of conditions favoring cooperation is extended with policing. In these simulations, we varied cell diffusion (0.0001 – 3.5 μm2/s in steps of 0.5), public good diffusion (1 – 7 μm2/s in steps of 1), toxin diffusion (1 – 7 μm2/s in steps of 1), and the toxin death threshold θT (1000 – 2200 molecules in steps of 400).
Simulating three-way interactions
Finally, we asked whether policing is an evolutionary stable strategy or whether it can be outperformed by a public good producing strain R that no longer produces the toxin, but is resistant to it. To address this question, we simulated three-way interactions between and W, C and R (across a range of diffusion conditions: Dc = 1 – 2 μm2/s in steps of 0.5, dp = 3 – 5 μm2/s in steps of 1). Because we do not necessarily expect an overall winner in these competitions, as strain frequency could potentially follow cyclical patterns [44], we extended our simulation code by implementing a random cell removal event, which is activated as soon as more than 30 % of the surface is covered with cells. This allowed us to keep population size at roughly K/2. In this set up, we followed strain frequency over time and arbitrarily stopped simulations after 80,000 time steps.
Acknowledgements
We thank Barbara König and Marta Manser for comments, the Services and Support for Science IT (S3IT) at the University of Zurich for technical support with the simulations on the ScienceCloud, and the Swiss National Science Foundation (grant no. PP00P3-139164) and the European Research Council (ERC-CoG grant no. 681295) for funding.