Abstract
Spatial structure greatly affects the evolution of cooperation. While in two-player games the condition for cooperation to evolve depends on a single structure coefficient, in multiplayer games the condition might depend on several structure coefficients, making it difficult to compare different population structures. We propose a solution to this issue by introducing two simple ways of ordering population structures: the containment order and the volume order. If population structure 𝒮1 is greater than population structure 𝒮2 in the containment or the volume order, then 𝒮1 can be considered a stronger promoter of cooperation. We provide conditions for establishing the containment order, give general results on the volume order, and illustrate our theory by comparing different models of spatial games and associated update rules. Our results hold for a large class of population structures and can be easily applied to specific cases once the structure coefficients have been calculated or estimated.
Introduction
The evolution of cooperation is a fascinating topic that has been studied from different perspectives and theoretical approaches (2, 14, 21, 36). An issue that has led to considerable interest is the extent to which spatial structure allows cooperation to thrive (1, 7, 10, 19, 20, 22, 27, 29⇓-31, 38-40, 46, 48, 51, 53, 55-60). Spatial structure can both enhance cooperation by inducing clustering or assortment (whereby cooperators tend to interact more often with other cooperators (11, 13, 22)) and oppose cooperation by inducing increased local competition (whereby cooperators tend to compete more often with other cooperators (47)). For two-player games or multiplayer games with similar strategies, the balance between these two opposing effects is captured by the “scaled relatedness coefficient” of inclusive fitness theory (31, 46, 59, 60) or the “structure coefficient” of evolutionary game theory (1, 38, 53). These coefficients are functions of demographic parameters, and take into account the degree of assortment, the effects of density dependence, and the strength of local competition resulting from spatial interactions (20, 31, 53). Two different models of spatial structure and associated evolutionary dynamics can be unambiguously compared by ranking their relatedness or structure coefficients: the greater the coefficient, the less stringent the conditions for cooperation to evolve. Hence, different models of population structure can be ordered by their potential to promote the evolution of cooperation in a straightforward way.
Despite the theoretical importance of models leading to a single relatedness or structure coefficient, many examples of social evolution ranging from microbial cooperation (18, 32, 63) to collective action in humans (23, 35, 43) involve games between more than two players with distinct strategies (17, 44). In these cases, the effects of spatial structure cannot be captured by a single coefficient, as higher degrees of association (e.g., “triplet relatedness” (39, 42)) are required to fully describe the condition under which cooperation is favored (49, 59, 62). The need to account for several structure coefficients has so far precluded a simple way of comparing population structures independently of the particular game used to model cooperation.
Here, we propose a framework to order population structures by their potential to promote cooperation that is also valid in the case of games between multiple players with distinct strategies. Our framework allows the comparison of two population structures without referring to any concrete game. We will distinguish two cases, depending on the inclusion relation between the sets of games for which cooperation is promoted under each population structure. (i) The set of games for which the second population structure promotes cooperation is fully contained in the set of games for which the first population structure promotes cooperation (Fig. 1A). In this case, we say that the first population structure is greater than the second in the containment order, and hence a stronger promoter of cooperation. (ii) The set of games for which one population structure promotes cooperation is not fully contained in the set of games for which the other population structure promotes cooperation (Fig. 1B). In this case, we say that the population structure promoting cooperation for a larger volume of games is greater in the volume order, and hence a stronger promoter of cooperation.
So far, the structure coefficients for general multiplayer games have been calculated only for few population structures, as such calculations often represent a technical challenge (34). However, once the structure coefficients are known, the containment and volume orders we propose here allow to assess the consequence of population structure on the evolution of cooperation independently of the game at stake. This way, our approach can help to organize myriads of results on the promotion of cooperation in spatially structured populations.
Methods and Results
Cooperation games and polytopes
We consider symmetric games between d players with two strategies, A and B. A focal player舗s payoff depends on the player舗s own strategy and on the strategies of its d – 1 co-players. If j co-players play A, a focal A-player obtains aj, whereas a focal B-player obtains bj. These interactions are represented by the payoff table:
It follows that a game is determined by 2d real numbers and can thus be considered as a point in a 2d-dimensional space.
In which sense can we say that one population structure favors cooperation more than another? To answer this question precisely, we first need to specify what we mean by “cooperation”, as this could refer to different social behaviors, in particular if we move beyond two-player games (25). We are interested in a particular subset of games that we call “cooperation games”. In these games, players decide whether to cooperate (play A) or defect (play B), and payoffs are such that: (i) players prefer other group members to cooperate irrespective of their own strategy, and (ii) mutual cooperation is favored over mutual defection. In terms of our payoff parameters, these conditions imply as well as Conditions (1) and (2) are often used to characterize the benefits of cooperation in multiplayer social dilemmas (25), such as the provision of collective goods (46). However, our conditions do not specify individual costs associated to a decision to cooperate, and hence our class of cooperation games includes not only social dilemmas, but also mutualistic games in which individual and group interests are aligned. If we further restrict payoffs to values between zero and one, then the set of all cooperation games with d players is given by a (convex) polytope (64) in a 2d-dimensional space, which we denote by P. A polytope is a geometric object with flat sides, the generalization of a polygon (which is a 2-dimensional polytope) to higher dimensional spaces. See Supplementary Material for further details.
We need to specify precisely what we mean by “favoring” cooperation. For our purposes, we say that cooperation is favored if a single cooperator in a population of defectors has a higher probability of eventually reaching fixation than a single defector in a population of cooperators (37). This also means that cooperation is more abundant than defection in a mutation-selection process in the limit of low mutation (16). For weak selection on homogeneous populations of constant size, strategy A is favored over B if (62) where σ0…,σd-1 are the d structure coefficients. These are independent of payoffs aj, and bj, but dependent on the type of spatial structure (for instance, where the co-players of a given focal individual are located) and update rule used to model the evolutionary dynamics. In Table 1, we provide examples of population structures and their corresponding structure coefficients (see Supplementary Material for a derivation).
The structure coefficients are uniquely determined up to a constant factor. Setting one of them to one thus gives a single nontrivial structure coefficient for two-player games (53). We use the sequence σ to collect the coefficients and note that, if σj ≥ 0 for all j and σj > 0 for at least one j, we can impose without affecting the selection condition (4). For our purposes, this normalization turns out to be more useful than setting one coefficient to one. In particular, such normalization allows us to understand the (normalized) structure coefficients as describing a probability distribution, and to make a straightforward connection with the concept of assortment as developed for the case of linear public goods games (3, 13). To do so, let us rewrite the selection condition (4) as Here, σj plays the role of the “effective” probability of interacting with j individuals of the own type (and d – 1 – j of the other type). As given by (5), the selection condition states that A is favored over B if the expected payoff of an A-player is greater than that of a B-player when the “interaction environments” (13) are distributed according to σ.
A given population structure will favor cooperation only for a subset of cooperation games. More precisely, for a population structure Si with structure coefficients σi, the set of cooperation games for which Si favors A over B is given by adding the selection condition (5) to the inequalities defining the polytope of cooperation games, P, i.e., (1), (2), and (3). The selection condition (5) defines a hyperplane and thus divides the space of games into two: those for which cooperation is favored and those for which defection is favored. This shows that our problem is equivalent to a geometric problem in 2d dimensions. In the following, we denote by Qi the polytope containing the cooperation games for which cooperation is favored under population structure Si (see Supplementary Material).
Containment order
If the set of games Q2 for which cooperation is favored under population structure S2 is contained in the set Q1 for which cooperation is favored under population structure S1, then we say that S1 is greater than S2 in the containment order (12), and we write . The ordering implies that cooperation cannot be favored under S2 without also being favored under S1.
Establishing the containment order is equivalent to a “polytope containment problem” (24), consisting on determining whether or not a polytope is contained in another. Polytope containment problems can be solved numerically by linear programming (15). Here, we describe an alternative and simpler approach borrowed from the literature on stochastic orders (50). First, assume that the structure coefficients σj are nonnegative and normalized, so that they define a probability distribution over j = 0, 1,…, d – 1. In this case, the left-hand side of the selection condition (4) can be interpreted as the expected value E[f(J)], where f(j) ≡ fj = aj – bd-1-j, and J is the random variable associated to the probability distribution σ. Consider now two population structures S1 and S2 with structure coefficients σ1 and σ2, and associated random variables J1 and J2, respectively. A sufficient condition leading to the containment order is hence that for all cooperation games.
In order to evaluate this condition, we make use of the usual stochastic order (50). A random variable J1 is said to be greater than J2 in the stochastic order if and only if for all increasing functions ϕ. This is denoted by . Conveniently, and by (1), the sequence fj is always increasing in j, allowing us to apply this idea directly (see Proposition 1 in Supplementary Material for details). One advantage of expressing the containment order in terms of the stochastic order is that we can transform our original polytope containment problem into the problem of finding conditions under which random variables can be stochastically ordered. Some of these conditions follow from a simple inspection of the sequences of structure coefficients. For instance, a sufficient condition leading to the stochastic order (and hence to the containment order ) is that σ1 – σ2 has exactly one sign change from – to + (50). As we show in Examples, this simple condition allows us to order different existing models of population structure in a straight forward way.
For the linear public goods game (i.e., a game with payoffs aj = β(j + 1) – c and bj = ßj for some β > γ > 0 where β is the marginal benefit from the public good and γ is the individual cost of contributing), the selection condition (5) can be put in a form reminiscent of Hamilton’s rule with (eA – eB) /(n – 1) playing the role of a measure of assortment (or relatedness), where is the mean number of cooperators among the d – 1 interaction partners of a cooperator (resp. defector) (3). For more general cooperation games, the selection condition depends not only on the mean but also on higher moments of the probability distribution given by σ. The stochastic order we have used for establishing the containment order is a way of measuring the association between strategies in this general case. Hence, it can be said that population structures greater in the containment order are those characterized by greater “effective assortment” and thus more conducive to the evolution of cooperation. In the extreme case where σd–1 = 1 (and σj = 0 for j ¹ d – 1), we have the case of a completely segregated population where As only interact with As and Bs only interact with Bs. In this case, the selection condition reduces to (2), and cooperation is always favored by definition.
It can happen that neither Q1 is entirely contained in Q2 nor Q2 is entirely contained in Q1. In these cases, S1 and S2 are incomparable in the containment order (i.e., neither nor hold) and we write . We show in Proposition 2 in Supplementary Material that a sufficient condition leading to such incomparability is that the sequences σ0 and σ2 cross twice (Fig. 2). In this case, there exist both a subset of cooperation games favored under S1 but not under S2 and a subset of cooperation games favored under S2 but not under S1.
For the commonly discussed case of two-player games in structured populations (53), the sequence σ consists of two elements: σ0 (usually set to one) and σ0 (usually denoted by σ and referred to as “the” structure coefficient). Since two sequences of two elements can only cross each other at most once, it follows that any two population structures can be ordered in the containment order if d = 2, i.e., the containment order is a total order for two-player games. Moreover, the containment order is given by the comparison of the structure coefficients σ, with larger σ leading to greater containment order. Contrastingly, for d ≥ 3 two sequences σ can cross twice. In this case, their respective population structures cannot be compared in the containment order: for multiplayer cooperation games and for the space of all possible population structures, the containment order is only a partial order (see Proposition 3 in Supplementary Material).
Volume order
In order to address the cases for which two population structures are incomparable in the containment order, we introduce the “volume order”. We say that S1 is greater than S2 in the volume order, and write , if where Vol (X) is the volume of polytope X. In other words, means that for a given d, cooperation is favored under S1 for a greater number of cooperation games than under S2. If two structures are ordered in the containment order so that , this implies that they are ordered in the volume order so that , but the converse is not true.
We find that the volume of all d-player cooperation games P is given by (Proposition 10 in Supplementary Material; Fig. 3): which decreases rapidly with the number of players d. For d = 2, this volume is equal to 5/24. In this case, the four payoffs a1, a0, b1, and b0 can be ordered in 4! = 24 possible ways, five of which satisfy inequalities (1) and (2), namely (i) b1 ³ a1 ³ b0 ³ a0 (prisoner’s dilemma), (ii) b1 ³ a1 ³ a0 ³ b0 (snowdrift game), (iii) a1 ³ b1 ³ b0 ³ a0 (stag hunt), (iv) a1 ³ b1 ³ a0 ³ b0 (harmony game), and (v) a1 ³ a0 ³ b1 ³ b0 (prisoner’s delight (4)). For large d, condition (2) becomes less important and the volume of cooperation games is approximately 1/(d!2), which is the volume of games satisfying conditions (1) and (3).
For some population structures, such as large well-mixed populations updated with a Moran process, the structure coefficients are symmetric, i.e., σj = σd–1–j for all j. For these cases, the fraction of cooperation games for which cooperation is favored becomes (Proposition 11 in Supplementary Material; Fig. 3). This fraction is equal to 3/5 for d = 2, and reduces to 1/2 in the limit of large d.
Examples
Let us now illustrate our approach with particular models of spatial structure and associated update rules. Consider first the baseline scenario of a well-mixed population of size N ³ d updated with a death-Birth (Moran) process (17, 37). In the death-Birth process, each time step one individual is chosen at random to die and another one is chosen proportional to its payoff to reproduce by making a copy of itself. We find that for any d ³ 2, well-mixed populations updated with a death-Birth process are ordered in the containment order with respect to the total population size N, such that larger populations are more conducive to multiplayer cooperation (Proposition 4 in Supplementary Material). Our result generalizes previous results for two-player games and multiplayer games with similar strategies according to which smaller population sizes are less conducive to cooperation because of the stronger local competition among cooperators (Ref. (53), Eq. 22; Ref. (60), Eq. B.1). In the limit of large N and by Eq. (9), well-mixed populations updated with a death-Birth process favor cooperation for exactly one half of all possible cooperation games.
Consider now the effect of introducing spatial structure while keeping the same update rule. One of the simplest spatial models is the cycle (10). It has been shown that cycles updated with a death-Birth process are better promoters of cooperation than well-mixed populations in the case of two-player games (19, 41), and for several examples of multiplayer social dilemmas (such as linear public goods games, snowdrift games, and stag hunt games) in the limit of large population size (61). Our theory allows us to extend these results to all multiplayer cooperation games and arbitrary population sizes. Indeed, we find that cycles are greater than well-mixed populations in the containment order for any given population size N (Proposition 6 in Supplementary Material). This implies that cycles are better promoters of cooperation than well-mixed populations for any cooperation game, any number of players d, and any population size N.
A second model of spatial structure for which structure coefficients are readily available is the group splitting model of Ref. (58). In this model, a finite population of size N is subdivided into m groups, which can grow in size and split with probability q when reaching the maximum size n. In the limit of rare group splitting (q ≪ 1), all groups are typically of the maximum size n and the structure coefficients can be calculated analytically for general d-player games (26). Consider well-mixed and group-splitting populations updated according to a death-Birth process. If the number of groups is greater than two, the group splitting model is greater than any well mixed population in the containment order (Proposition 7 in Supplementary Material). Moreover, in the limit of m ≫ n, the structure coefficients of the group splitting model become σd–1 = 1 and σj = 0 for j ≠ d – 1. In this limit, the group splitting model is greater in the containment order than any other population structure. Hence, it is the population structure that favors cooperation most among all theoretically possible population structures.
The cycle and the group splitting model are better promoters of cooperation than the well-mixed population. But which one promotes cooperation under more cooperation games, the cycle or the group splitting model? Consider cycles of size N and group splitting models with rare group splitting (q ≪ 1) consisting of m groups of maximum size n, so that the total maximum population size is equal to N = mn. Assuming that the population size N is large, the containment order depends on the number of groups m of the group splitting model in the following way (Proposition 8 in Supplementary Material). (i) If the number of groups is small (m ≥ (n + 4d – 6)/(2d – 3)) the group splitting model is smaller than the cycle in the containment order. (ii) If the number of groups is intermediate (n + 4d – 6)/(2d – 3)) < m < n + 2) the group splitting model and the cycle are incomparable in the containment order. (iii) If the number of groups is large (m ≥ n + 2) the group splitting model is greater than the cycle in the containment order. As a particular example, consider a cycle of size N = 1000 and a group splitting model with m = 10 groups of maximum size n = 100 (Fig. 4). In this case, the cycle is greater than the group splitting model in the containment order if d ≤ 7, while the two population structures are incomparable in the containment order if d ≥ 8. Concerning the volume order, exact computations and numerical simulations suggest that the cycle is greater than the group splitting model for d ≥ 12, and smaller than the group splitting model otherwise.
Up until now we have compared different models of spatial structure (the well-mixed population, the cycle, the group splitting model) with a single update rule (the Moran death-Birth process). However, the structure coefficients depend both on spatial structure and on the update rule. For two-player games, different update rules can have important consequences on the evolutionary dynamics, as they lead to different “circles of compensation”, or how far the effects of density dependence extend from a given focal individual (20). What are the effects of different update rules on multiplayer cooperation games? As an example, consider well-mixed populations with two different update rules: the Moran process, where a random individual dies and its neighbors compete for the empty site, and the aspiration dynamics, where an individual is likely to switch its strategy if the current payoff does not meet an aspiration level (8, 52). The two update rules can be ordered in the containment order only if 2d–1 (N–d) ≤ d(N–1) (Proposition 9 in Supplementary Material). In this case, aspiration dynamics is greater in the containment order than the Moran process, meaning that if cooperation is favored under the Moran process it will also be favored under aspiration dynamics, but not necessarily vice versa. If 2d–1 (N – d) > d(N – 1) the two structures are incomparable in the containment order. However, for any finite population size N, aspiration dynamics is greater in the volume order: overall, cooperation is favored for more games under aspiration dynamics than under the Moran process.
Discussion
Our approach to compare models of population structure sheds new light on how to study and analyze the evolution of cooperation in spatially structured populations. We have shown how several existing results, obtained under the assumptions of pairwise interactions, similar strategies, or particular classes of multiplayer social dilemmas, generalize to the case of multiplayer cooperation games with distinct strategies that we have considered here. Perhaps more importantly, one can find two population structures such that there is a class of cooperation games for which cooperation is favored under the first but not under the second, and a class of cooperation games for which the opposite holds true (Fig. 1B). Thus, arbitrarily choosing one or a few games from the set of all possible cooperation games to compare the effects of population structure on the evolution of cooperation can be misleading, even when focusing on the comparison of fixation probabilities under weak selection. This is different from the case of either two-player games or multiplayer games with similar strategies, where a ranking of population structures is always possible in terms of a single real value, and where it is sufficient to focus on a single game without loss of generality (31, 53).
We made use of the theory of stochastic orders (50) to provide conditions under which two population structures are comparable or incomparable in the containment order. Within social evolution theory, stochastic orders have been also recently used to tackle the question of whether variability in the group size distribution would lead to less stringent conditions for the evolution of cooperation in multiplayer social dilemmas (45). Our use of stochastic orders in this paper relies on the assumption (fulfilled by all the population structures we used as examples) that the structure coefficients can always be normalized to define a probability distribution. It would be interesting to investigate under which general conditions such assumption is valid. Another open question is whether two population structures incomparable in the containment order could favor cooperation in disjoint subsets of cooperation games. If the structure coefficients define a probability distribution, this will never be the case, as it will always be possible to find a cooperation game for which the selection condition holds for any two population structures. Consider for instance a game for which aj = α and bj = 0 for all j, with α > 0 (a mutualistic game where the group optimal action A is also individually optimal). In this case, and provided that the structure coefficients are nonnegative, the selection condition (4) is always satisfied.
We considered a very broad definition of cooperation and a particular measure of evolutionary success, and investigated subset containment relations and volumes of the resulting polytopes. In this respect our approach is related to a classical study by Mattesiand Jayakar (33), who first defined “analtruism domain” from a set of linear inequalities involving “local fitness functions” and then investigated the problem of finding and measuring the relative volume of the “subset of the altruism domain in which A is more fit than B on average, that is, altruism can evolve”. We note, however, that our definition of cooperation is different from the definition of altruism adopted by Matessiand Jayakar: the “multi-level interpretation” of altruism, in the sense of Kerr et al. (25). In particular, we only focused on the group benefits, not the individual costs, associated to expressing the cooperative action A. Such costs could be introduced by adding further sets of inequalities to the ones we used here, for instance by requiring that aj ≤ bj for some or all j (25, 46). Since we did not specify any costs, our class of cooperation games contains a relatively large set of mutualistic games for which group beneficial behaviors are also individually beneficial. Our measure of evolutionary success is also different, as we focused on the comparison of fixation probabilities in the limit of weak selection, whereas Matessiand Jayakar focused on the one-step change in frequency. Finally, Matessiand Jayakar limited themselves to “linear fitness functions” (equivalent to linear games in our setup) while we considered more general multiplayer games. The differences between our study and the one by Matessi and Jayakar pinpoint possible future work along these lines. For instance, alternative definitions of cooperation that take in consideration the cost of cooperation (6, 25) and exclude mutualistic games could be explored, possibly together with alternative measures of evolutionary success (54). As long as it is possible to write all conditions as a set of linear inequalities (and hence as polytopes) involving the payoffs of the game, our definitions can be used and adapted to these cases. It would be interesting to see the extent to which comparisons of different population structures based on the containment and volume orders defined here are robust to changes on the way cooperation and evolutionary success are defined and implemented.
Appendix
Computing volumes
There are many exact methods for computing volumes of polytopes, including triangulation methods (5) and signed decomposition methods (28). Computing the exact volume of a polytope is however known to be #P-hard (9), and a simple task only for low dimensions. We calculated exactly the volumes in Fig. 4 for d ≥ 6 using the function volume of the class Polyhedron of the mathematics software Sage (version 6.5). For d ≤ 7, we used a Monte Carlo method for approximating the volumes. For each value of d, we randomly generated 106 increasing sequences aj, and bj, and retained only those which fulfilled (2). Wethen checked how many of these sequences verified the selection condition (4). The fraction of cooperation games was then approximated by the ratio between these two numbers. Our source code in Python is publicly available on GitHub (https://github.com/jorgeapenas/ordering).
Supplementary Material
Supplementary Methods: Ordering structured populations in multiplayer cooperation games (PDF).
Acknowledgments
We would like to thank Laurent Lehmann, Georg Noldeke, and three anonymous referees for their valuable comments on previous versions of this paper.
Authors’ Contributions
J.P. designed the study and wrote the code; J.P. and B.W. performed research; all authors wrote the manuscript.
Funding Statement
J.P. acknowledges funding support by the Swiss National Science Foundation (grant PBLAP3-145860).
References
- [1].↵
- [2].↵
- [3].↵
- [4].↵
- [5].↵
- [6].↵
- [7].↵
- [8].↵
- [9].↵
- [10].↵
- [11].↵
- [12].↵
- [13].↵
- [14].↵
- [15].↵
- [16].↵
- [17].↵
- [18].↵
- [19].↵
- [20].↵
- [21].↵
- [22].↵
- [23].↵
- [24].↵
- [25].↵
- [26].↵
- [27].↵
- [28].↵
- [29].↵
- [30].↵
- [31].↵
- [32].↵
- [33].↵
- [34].↵
- [35].↵
- [36].↵
- [37].↵
- [38].↵
- [39].↵
- [40].↵
- [41].↵
- [42].↵
- [43].↵
- [44].↵
- [45].↵
- [46].↵
- [47].↵
- [48].↵
- [49].↵
- [50].↵
- [51].↵
- [52].↵
- [53].↵
- [54].↵
- [55].↵
- [56].
- [57].
- [58].↵
- [59].↵
- [60].↵
- [61].↵
- [62].↵
- [63].↵
- [64].↵
References
- [1].
- [2].
- [3].
- [4].
- [5].
- [6].
- [7].
- [8].
- [9].
- [10].
- [11].
- [12].
- [13].
- [14].
- [15].
- [16].
- [17].
- [18].