Abstract
Inter-cellular interactions are ubiquitous in the world of microbes, shaping the population composition of ecosystems at both microscopic and macroscopic scales, affecting human health and governing processes in utilization of bio-resources. However, metabolite exchanges, a major type of microbial interactions, remain difficult to measure and predict, invoking the urgent need of modeling and computational studies. As an alternative to the conventional ecological models which usually lack metabolic details, metabolic models and flux-balance-analysis (FBA) based algorithms emerge as a promising way to address the challenge. However, existing algorithms for predicting microbial community metabolism usually impose constraints or objective functions (implicitly or explicitly) that lead to ‘forced altruism’, which forces a microbe to fulfill other species’s need by cross feeding certain metabolites instead of using the resource for its own reproduction and other cellular activities in order to achieve community level optimality. As a result, in terms of game theory, the prediction is not necessarily a Nash equilibrium and therefore not evolutionarily stable. We developed a bi-level optimization framework free of ‘forced altruism’ constraints termed NECom. Payoff matrices of metabolic strategies analogous to traditional matrix games can be obtained by shadow price analysis in FBA to validate NECom predictions. By applying NEcom to toy community models, we demonstrate several classical games between microbes in terms of metabolic interactions, including prisoner’s dilemma and positive frequency-dependent cooperation. The results provide insights into why microbes may not prefer cooperation even if it is mutual beneficial, and why sometimes mutualism is still favorable when the resource investment seemingly contradicts to a microbe’s fitness, demonstrating NECom a promising tool to reveal metabolic mechanisms of microbial interactions. The novel tools reported in this paper bridge traditional evolutionary game theory and metabolic models for better analysis of microbial metabolic interaction.
Introduction
In nature, microorganisms seldom exist in isolate form. Instead, they form communities governed by different types of interactions, which play essential roles in adaptation to environment [13, 37, 47] and evolution of species [8, 28, 29, 40, 49]. Beside the theoretical importance, microbial interactions are also a key topic in applications from medicine intermediate synthesis [7, 64] to remediation of gut microbiome for human health [17, 33, 53, 55], from degradation of cellulose for biorefinery [34, 43, 69] to microbial power generation [38]. Among microbial interactions, exchange of metabolites is especially important and arguably responsible for the fact that more than 99% of bacterial species are not cultivable [31, 47, 50]. Understanding metabolic interactions is a fundamental task in life science. Despite continual progresses in determining metabolite exchanges using experimental approach such as spatially separated apparatus designs [44, 63], isotope probing [1, 59] and tracing [16] techniques, 16s RNA analysis [62] and metabolome analysis [42], we still need governing principles to predict and understand these metabolic interactions.
Benefiting from the advancement in genome-scale metabolic models [36], the application of constraint-based modeling to microbial communities have started since a decade ago [54]. Solutions have been proposed while new challenges emerged in the process. Joint flux balance analysis (Joint-FBA) was first introduced to model a microbial community as a ‘super’ organism containing the compartments for each organism and an additional compartment for inter-cellular metabolite exchange [14, 54]. The total community biomass was maximized to predict metabolism in Joint-FBA. Methods based on Joint-FBA for analyzing host-microbe and microbe-microbe interactions were also developed and widely applied to the gut microbiome [22–24, 39]. Later FBA were adapted for modeling dynamic metabolism of microbial co-culture [20, 65]. Succeeding efforts include seeking better ways to select particular solutions among equivalent optima in each time step to ensure a well-conditioned dynamical system [18, 60] and incorporating spatiotemperol elements [3, 21, 25, 26]. In order to reconcile the community and individual objective functions, OptCom, a bilevel optimization algorithm was developed to optimize the individual fitness in the inner problem, while optimizing community fitness in the outer problem [66]. To address the effect of viable abundance on the inter-cellular fluxes at population steady state, community FBA (cFBA) [35] generalized joint-FBA by adding parameterized abundances as weights to exchange fluxes. SteadyCom [6] as a reformulation of cFBA enabled efficient computation for flux variability analysis and other constraint based modeling techniques for communities with a large number of organisms. In recent years, researchers have started to simulate communities with a large number of species [27, 52] or handle a large quantity of scenarios [46, 61, 67]. Joint-FBA and its linear derivatives are usually employed to avoid intractable computation. However, in most static FBA-based algorithms, some constraints and the community objective function might force individual organisms to produce a certain amount of metabolites for other members prior to optimizing its own fitness whenever doing so the community level objective is optimal. We call this ‘forced altruism’. We believe that forced altruism is potentially applicable to microbes that are physically connected, e.g., by nanotubes [9, 48] or to endosymbiosis, since inter-cellular exchange under these situations are likely governed by concentration gradients or regulation by the host, respectively. In general for microbes in a co-culture that are separate from each others by cell envelopes and are not controlled by higher level regulatory mechanisms (e.g., host-regulation, quorum sensing etc), however, the applicability of ‘forced altruism’ constraints should be questioned because microbe cells with higher individual fitness would usually be favored by selection. Consequently, as a necessary condition for evolutionary stability, true Nash equilibria (NE) in community metabolic networks should be the targets to identify, in which each microbe maximizes its fitness function in the given environment. In context of community metabolic networks, the microbial game is significantly different from those studied previously which treat a single microbial cell as the basic unit with ad-hoc metabolic details [2, 4, 5, 15, 19, 30, 32]. Instead, in community metabolic networks, each microbe is represented as the sum of its constituent biomacromolecules synthesized from the network. The available strategies for each player are any possible flux distributions (the entire set of reaction fluxes) satisfying biochemical principles and constrained by substrate availability, which depends not only on the nutrients in the environment but also the strategies of other players in terms of the cross feeding metabolites they may produce [51]. The metabolic details, continuous flux space and interdependence of available strategies between players characterize a unique class of microbial games in community metabolic networks which require novel game-theoretical methods.In this paper we firstly introduce a method called ‘FShap’, to construct the payoff matrices of two interactive species with the solution of Flux Balance Analysis, so that the Nash equilibria can be obtained by traditional payoff matrix analysis, then We introduce a bi-level mixed integer optimization framework free of forced altruism termed NECom. Based on toy models representing classical games in the context of community metabolic networks, the predictions of FShaP and NECom will be compared. The strategies predicted by NECom will be proved to be Nash equilibria (NE) and the community objective function can be set to find potential strong Nash equilibria and Evolutionary Stable Strategies (ESS), a NE not invadable by alternative strategies.
Result and Discussion
Developing FShaP, the FBA Shadow price based Payoff matrix
In traditional game theory, payoffs for each species involved in the metabolic inter-cellular interactions can be computed as the benefits of importing metabolites subtracting the costs of exporting metabolites.But how can we use metabolic model to obtain the benefits and costs? The answer was found in the solution of flux balance analysis(FBA) [45], specifically, the shadow price of lower bound constraint of exchange reactions. Before recalling the FBA algorithm, let N be the set of organisms in a microbial community, for each organism n ∈ N, let In be the set of metabolites and Jn be the set of reactions, cn be the objective indicator vector, in which 1 indicates the corresponding flux variable vj,n ∈ Vn of reaction j is the objective and 0 for otherwise. where Si,j,n is the stoichiometry of metabolite i for reaction j, LBj,n and UBj,n are the lower bound and upper bound for the flux of reaction j, respectively. All subscripts n stand for organism n.From the primer problem, the dual problem of FBA can be derived as: Where µLB and µUB are the shadow price for the lower bounds constraints eq. (2) and the upper bounds constraints eq. (3) respectively, λn are the dual variable for eq. (1).
By the definition in linear programming, shadow price of a constraint is how the optimal value of objective changes in response to marginal variation of the right-hand-side of the constraint. Therefore as to a specific exchange reaction com(i, n) ∈ Jex, the shadow price is the amount of fitness increases(or the benefit) of organism n in response to one (marginal) unit increase of −LBcom(i,n),n,the maximum uptake rate of metabolite i by organism n, can also been regarded as the cost when organism n export metabolite i. Considering a classical two-member game, the payoff matrix fig. 1 can be constructed with the payoffs calculated with the cost and benefit obtained from the solution of dual problem of FBA, using eq. (5) and eq. (6)
Given specific metabolite exchange state, payoff values for individual organism in a the two-member game can be computed as: where S1, S2 are strategy indicator vectors for the two species respectively, with 1 for activation and 0 for inactivation (e.g S12 = [1, 0] means the strategy combo No.2 for species 1 is activated for metabolite 1 but not for metabolite 2, and S1i,2 is the strategy indicator for metabolite i in S12).
Steps to develop NECom, the Nash Equilibrium predictor for microbial Community)
In this section we derive the bi-level optimization framework NECom, starting with the description of flux balance analysis (FBA) as the inner problem for optimizing the fitness of individual organisms. A set of outer-level variables to indicate the availability of metabolites for uptake are then introduced to connect the inner-level flux variable.
Let be the set of extracellular metabolites in the community, be the set of exchange reactions between the community and organism n. Define an index mapping functions such that map extracellular metabolite to its corresponding exchange reaction. For each individual organism, we split the exchange reaction flux for a possible cross-feeding metabolite vcom(i,n),n into two non-negative continuous variable: the uptake rate and the export rate : To explicitly model the mutual dependency in terms of inter-organism metabolite exchange, uptake of a metabolite by an organism is possible only if there is surplus of the metabolite in the medium after the consumption/production by the rest of the community: where Xn is the relative abundance of organism n (a pre-set parameter), which is multiplied by specific uptake rate to correctly normalize the exchange with the community, and is the maximum community uptake of metabolite i from the medium. Here to avoid forced altruism, is designated as an inner variable independent of the outer problem, while the uptake rate is also an inner variable but depends on the net availability of metabolite i. In this way we can ensure that individual organisms have ‘autonomy’ over their metabolite exports. Maximization of the community fitness in the outer level cannot force an organism to produce a certain metabolite unless FBA determines that the metabolite is necessary for, or at least not undermining the maximum growth of the organism. The max function in the eq. (7) is linearized by introducing an outer continuous variable βi,n and an outer binary variable δi,n: where M is a large constant. In combination of inner problem and outer problem the complete formulation of NECom can be written as: For the purpose of finding the ESS or strong Nash equilibria, an community level objective function can be added, usually it is the sum of biomass of all community members: , and this community level objective is added to NECom for the predictions in this paper.
NECom prediction can guarantee Nash Equilibrium
In the context of microbial games, Nash equilibrium (NE) is a state where there is no benefit for any community member to unilaterally deviate from its current strategies [5, 68]. To prove whether a NECom prediction can guarantee Nash equilibrium, firstly one member is picked and the fluxes(strategies) of other members are fixed, so that any deviation of current flux distribution of the picked member can be ‘unilateral’,then we check if NECom predicted flux distribution for the picked member can be optimal, taking steps along these paths, we have the following proof:
According to its inner problems of the NECom framework, when the strategies of other members are fixed, any member k ∈ N in the community model has the following optimization formulation: If metabolites i can not be net synthesized by other species, then according to eq. (8), βcom(i,k),k is 0, and eq. (16) can be reduced to Obviously the above constraints do not have any effect on flux distribution of member k.
On the other hand if metabolites i can be net synthesized by other species, then according to eq. (8),βcom(i,k),k is a fixed positive value, Xk is also fixed, is a non-negative variable that does not present elsewhere, therefor eq. (16) becomes effectively equivalent to which are simple substrate availability constraints that do not fix any exchange flux. in either aforementioned NECom is able to predict the optimal strategies(flux distributions) of member k, while it preserves its autonomy in an environmental conditions shaped by other species. now we proved that NECom predictions can guarantee Nash equilibrium.
Application of NECom to classical Games in Ecology
Consider scenario 1 where a simplest toy community model consist of two individual species (or mutants) with equal abundance named sp1 and sp2 (fig. 2). Each species can yield ATP through converting substrate S (assumed to be sufficient) to intermediate P and subsequently consume ATP though converting P to biomass precursors A and B, which are freely exchanged between the two species. The difference between them is that sp1 produces B more efficiently (consuming 1 ATP vs. 3 ATP for sp2), while sp2 produces A in more efficiently (consuming 1 ATP versus 3 ATP for sp1), reflecting the existence of variable pathway yields commonly observed in nature. Intuitively the optimal strategy for the community is that each species cross feeds each other with the metabolites produced with higher efficiency, i.e sp1 supplies B to sp2,while sp2 feeds A to sp1, coinciding with the prediction of joint-FBA and OptCom (they predict same results in this case, see (fig. 2 a). However, it is not a NE since either species, say sp1, can have a mutant that does not excrete B but keeps consuming A and increases the overall biomass production. In contrast, NECom predicts that the two species will grow at a lower rate and exchange nothing (fig. 2.b). Since there is no way for each species to grow faster given no export by another species, this is a NE. In analogy to classical matrix game, the payoff matrix corresponding to different crossfeeding states(fig. 3: a) was constructed with FshaP method, the crossfeeding state will migrate from one to another according to the current payoff matrix,which may also be updated after the migration. The payoff matrix indicates that maximum growth without cross feeding ([00, 00]) is the (strict) NE as well as evolutionary stable strategy (ESS), which is a NE not invadable by other mutants (strategies). Any other strategy combinations including the complementary crossfeedings ([01,10]) predicted by Joint-FBA/OptCom are not stable and will eventually evolve to the NE, no matter what the current crossfeeding state might be,it will eventually migrate to non-crossfeeding state since which is stable (fig. 3.b).
based on the analysis above, the interaction in (fig. 2) is characterized as a game called ‘prisoner dilemma’, because the NE is all members choosing ‘defect’ instead of ‘cooperate’, which NECom correctly predicts. The ‘forced altruism’ setup is the cause for the mutualistic prediction by joint-FBA and OptCom, because the setup artificially prioritizes the metabolite export by each member for its partner over the optimization of its fitness, in order to achieve higher community-level fitness.
In the previous toy community both members are self-sufficient, what if members must depend on each other for survival? In scenario 2, sp2 is set to depend on sp1 for its removal of the growth-inhibitory product ‘C’, which can not be withdrew efficiently from the system but needed by sp1 to generate ATP for growth (fig. 4). This model is an epitome of the interaction between M. maripaludis and D. vulgaris [57, 58], in which Methanococcus consumes the growth-requiring hydrogen produced by and meanwhile inhibitive to D. vulgaris. The NE predicted by NECom shows that sp2 will produce C as much as possible and both species can grow. The case demonstrates a type of mutualism resulting from coupled growth.
It is intuitive that the previous scenario will result at mutualism. What if the metabolite production is not growth coupled? In scenario 3, we use the same setup as in scenario 1 but remove the reaction for synthesizing metabolite A in sp1 and the reaction for synthesizing B for sp2 (fig. 5 a), then would cooperation become favorable or would defect still be the outcome despite no growth? This mutualist-cheater scenario is a classical question of interest in ecology [10–12, 41, 56]. The payoff matrix (fig. 5 b) shows that four strategy combinations (i.e the mutualism, two commensalisms, and non-crossfeeding) are NE, but mutualism is the ESS and there is no strict NE. Therefore from evolutionary game theory, the non-producing cheaters in this case can still co-exit with producer although not being able to outgrow them. This is another example to explain why mutualism or commensalism are possible even if it costs members some resource: because when a particular metabolite is not growth limiting, exporting the metabolite does not penalize the organism’s fitness.
Conclusion and outlook
For the purpose of seeking the ruling principles that governs the microbial inter-cellular interaction, we developed an bi-level algorithm called ‘NECom’ to predict the Nash equilibrium of multiple species’ interactive game at metabolic level. In order to compare NECom prediction with those obtained from traditional game theory analysis, a method termed ‘FShaP’ is proposed to construct the payoff table of inter-species metabolic interaction with the shadow price obtained in FBA solution of metabolic model, providing with a simple toy community model, NECom is proved to guarantee its predicted microbial interactions being Nash equilibria that previous methods’ predictions cannot guarantee, the main reason for this difference is that NECom does not contains ‘forced altruism’ setup that computationally force individual species fulfill other species’s metabolic need before its own. Being consistent with the predictions obtained by traditional game theory approach, NECom is demonstrated to be able to predict classical games including prisons’ dilemma and cooperative games.
The two method proposed in the paper, FShaP and NECom are functional complementary, although FShaP can be generalizable to more than two exchanging metabolites, however with increasing exchanging metabolites, the solution space will explosively increase, causing difficulty/inefficiency in enumeration of all Nash equilibria and finding strong Nash equilibria and ESS, especially when genome scale metabolic models are used in the computing. NECom, utilizing modern mixed integer programming technology, is able to efficiently search Nash equilibria for large scale and complex systems, however it does not predict the strategies’ migration(might be called ‘co-evolution’) path that can be reveal by FShap approach, these two method are developed to bridge traditional evolutionary game theory and metabolic models for in-depth analysis of inter-cellular interactions.
In the future, FShaP and NECom will be applied to some real-world case studies, higher level microbial interactions such as quorum sensing and host-microbe signaling can be considered in the form of additional constraints, kinetic parameters can be added and NECom can be adjusted for spatial-tempo simulation, various inner problem objectives can be evaluated for specific case studies. more comprehensive and better refined model can be used to improve prediction quality if computational difficulties can be well handled, multi-level omics data can be integrated in the framework to improve case specific predictions.
Author Contributions
Supervision: Tianwei Tan and Siu Hung Joshua Chan
Conceptualization: Jingyi Cai and Siu Hung Joshua Chan
Methodology: Jingyi Cai and Siu Hung Joshua Chan
Hardware provision: Tianwei Tan
Visualization: Jingyi Cai
Writing-original draft: Jingyi Cai and Siu Hung Joshua Chan
Writing-review and editing: Siu Hung Joshua Chan, Tianwei Tan and Jingyi Cai
Acknowledgments
We acknowledge Dr. Wenyan Yuan, Dr. Guang Cheng, Dr.Xueying Ni and Ying Zhu for their useful comments