Evolutionary prisoner's dilemma games coevolving on adaptive networks
Hsuan-Wei Lee, Nishant Malik, and Peter J. Mucha
TL;DR
This paper models the coevolution of strategies and network structure in Prisoner's Dilemma games, demonstrating the effectiveness of approximate master equations in predicting system behavior and exploring different partner-switching dynamics.
Contribution
It introduces an approximate master equation framework for analyzing coevolving adaptive networks in Prisoner's Dilemma games, improving upon existing pair approximation methods.
Findings
Approximate master equations accurately predict simulation results.
System evolution and stationary states depend on partner-switching rules.
Different model variants show qualitative differences in utilities and dynamics.
Abstract
We study a model for switching strategies in the Prisoner's Dilemma game on adaptive networks of player pairings that coevolve as players attempt to maximize their return. We use a node-based strategy model wherein each player follows one strategy at a time (cooperate or defect) across all of its neighbors, changing that strategy and possibly changing partners in response to local changes in the network of player pairing and in the strategies used by connected partners. We compare and contrast numerical simulations with existing pair approximation differential equations for describing this system, as well as more accurate equations developed here using the framework of approximate master equations. We explore the parameter space of the model, demonstrating the relatively high accuracy of the approximate master equations for describing the system observations made from simulations. We…
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 11
Figure 11
Figure 6
Figure 6
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17
Figure 18
Figure 19
Figure 20
Figure 21
Figure 22
Figure 23
Figure 24
Figure 25Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Evolutionary prisoner’s dilemma games coevolving on adaptive networks
\nameHsuan-Wei Lee
\nameNishant Malik
∗Corresponding author: [email protected]
Department of Sociology, University of Nebraska–Lincoln
Department of Mathematics, Dartmouth College
\namePeter J. Mucha
Department of Mathematics, University of North Carolina at Chapel Hill
Abstract
We study a model for switching strategies in the Prisoner’s Dilemma game on adaptive networks of player pairings that coevolve as players attempt to maximize their return. We use a node-based strategy model wherein each player follows one strategy at a time (cooperate or defect) across all of its neighbors, changing that strategy and possibly changing partners in response to local changes in the network of player pairing and in the strategies used by connected partners. We compare and contrast numerical simulations with existing pair approximation differential equations for describing this system, as well as more accurate equations developed here using the framework of approximate master equations. We explore the parameter space of the model, demonstrating the relatively high accuracy of the approximate master equations for describing the system observations made from simulations. We study two variations of this partner-switching model to investigate the system evolution, predict stationary states, and compare the total utilities and other qualitative differences between these two model variants. Evolutionary games; prisoner’s dilemma; adaptive networks; network dynamics; approximate master equations
1 Introduction
Game theory is the study of strategic decision making and the analysis of mathematical models of conflict and cooperation between intelligent rational participants harsanyi1988general ; myerson1991game ; gibbons1992primer . Game theory is used in a variety of disciplines, including biology, economics, political science, computer science, and psychology. In classical game theory, rational actors make their choices to optimize their individual payoffs. That is, actors make strategic choices on a rationally determined evaluation of probable outcomes, or utility, considering the strategic analysis that the players’ opponents are making in determining their own choices. Evolutionary game theory smith1982evolution ; nowak1993spatial ; weibull1997evolutionary ; hofbauer2003evolutionary ; hauert2006evolutionary , the application of game theory to evolving populations, has recently expanded from consideration of lifeforms in biology to various areas in social science. Evolutionary game theory is useful in this context by defining a framework of contests, strategies, behaviors, and analytics in which competition can be modeled. The key point in evolutionary game theory is that the success of a strategy is not just determined by how good the strategy is in itself; rather, it is a question of how good the strategy is in the presence of other alternative strategies and of the distribution of those other strategies within a population.
One of the great difficulties of Darwinian theory, recognized by Darwin himself, was the problem of altruism mitteldorf2000population ; in particular, exploited cooperators are worse off than defectors. Hence, according to the basic principles of Darwinian selection, intuitively it seems almost certain that cooperation should go extinct. If the basis for selection is at the individual level, the phenomenon of altruism is often hard to interpret. And selection at the group level, or for the “greater good,” appears to violate the game theory assumption of individuals maximizing their own utility. Indeed, altruism is certainly not found to be the general case in nature. Nevertheless, altruistic behaviors can be found in many social animals and, indeed, can be fundamental for some species to survive fletcher2007evolution .
The solution to this apparent paradox can be demonstrated in the application of evolutionary game theory to the prisoner’s dilemma game miller1996coevolution ; axelrod2006evolution , a game which tests the outcomes of “cooperating” versus “defecting.” Cooperation is usually analyzed in game theory by means of a non-zero-sum game. The prisoner’s dilemma game, one of the most studied systems in all of game theory, is a standard example that shows why two completely rational individuals may not cooperate, even if it appears that it is in their best collective interest to do so. Within evolutionary game theory, the analysis of the prisoner’s dilemma is as an iterative game mertens1989repeated , with the repetitive nature affording competitors the possibility of retaliating or defecting based on the results of previous rounds of the game. There are a multitude of strategies which have been tested by the mathematics of evolutionary game theory and in computer simulations of contests (see, e.g., fowler2008tournament ), with the general conclusion that the most effective competitive strategies are typically cooperative with a reserved retaliatory response as necessary. The most famous and one of the most successful of these strategies is “Tit for Tat,” which carries out this approach by executing a simple algorithm axelrod2006evolution ; nisan2007algorithmic .
In recent years, games played on various random graphs and social networks have been investigated nowak1992evolutionary ; killingback1996spatial ; abramson2001social ; ebel2002coevolutionary ; jackson2002formation ; hauert2004spatial ; lieberman2005evolutionary ; santos2006cooperation ; fu2008reputation ; fu2009partner ; hidalgo2015cooperation ; pinheiro2016linking . For example, the public goods game can be seen as a generalization of multiplayer N-player games and group interactions such as pattern formation and self-organization could be studied through this extension hauert2002replicator ; szolnoki2009topology ; perc2013evolutionary . The studies devoted to evolutionary games on complex networks are extensive. Speaking generally, the goals of such studies typically include identifying which combination of game rules, dynamics, and various network topologies can provide cooperation among selfish and unrelated individuals. In the present work, we focus on the dynamical role of players having the option to switch partners, as studied in the framework of adaptive networks that coevolve with the player strategies pacheco2006active ; pacheco2006coevolution ; szolnoki2008making ; szolnoki2009resolving ; szolnoki2009emergence ; perc2010coevolutionary . On a “coevolving” or “adaptive” network with a game, the vertices represent players and the edges denote the pairings, or game interactions, between players.
Importantly, the spatial structures in a network could enable cooperators to form small groups or clusters to protect themselves against exploitation by defectors nowak1992evolutionary . Studies have shown that the spatial structure can promote cooperation kerr2002local , but there is also evidence that suggests that spatial structure may not necessarily favor cooperation hauert2004spatial . Other studies have elaborated on different aspects of cooperation on scale-free gomez2007dynamical ; pusch2008impact ; perc2009evolution , square-lattice perc2008restricted ; roca2009promotion , small-world kim2002dynamic ; santos2005epidemic ; szolnoki2009impact , social and real-world networks holme2003prisoners ; fu2007social ; lozano2008mesoscopic , and multilayer networks wang2015evolutionary . Insofar as the player pairing network is part of a larger social setting, other notable rules encouraging cooperative behavior are kin selection hamilton1964genetical , group selection dugatkin1996cooperation ; traulsen2004stochastic , direct reciprocity pacheco2008repeated , indirect reciprocity nowak1998dynamics ; nowak2005evolution , social diversity perc2008social ; santos2008social ; yang2009diversity , voluntary participation hauert2002volunteering ; hauert2007via ; chen2008emergence , reputation fu2008reputation , and frequency of breaking cooperator-defector bonds and rewiring szolnoki2008making ; szolnoki2009resolving . All of these have been studied as interesting mechanisms that may promote cooperation in evolutionary games.
In this paper, we focus our study on a simple coevolving network model for how players play a prisoner’s dilemma game with the ability to adapt by changing their strategies or switching partners if they are exploited by their neighbors, introduced in fu2009partner . We are interested in exploring the cooperative level among the individuals as they organize into networks of cooperators. By using the technique of approximate master equations, we provide a more accurate approximation of the evolution of the network structure and player states with which to explore the parameter space of the model. We compare the existing analytical methods and our new approximation, providing qualitative and quantitative estimations of various network properties. In Section 2, we introduce the model. We then describe our more accurate approximation in Section 3. In Section 4, we provide numerical results and compare the two approximation methods. In Section 5, we introduce a variation to the model and compare the results of the two variants of the model. Finally, we make concluding remarks in Section 6.
2 Description of the CD-switching model
We study the partner switching model introduced by Fu et al. fu2009partner , where vertices of the coevolving network represent players, edges denote the pairing of individuals playing the game, and players adapting by changing strategies based on their current local information or switching partners. For simplicity, we initialize our simulations with an Erdős-Rényi network with nodes and edges, with each node assigned an initial state either as a cooperator () or defector (). Each node uses a single strategy across all of its links, though that strategy can change over time as we describe below. That is, individual playing with all of her connections obtains an income
[TABLE]
where is the neighborhood set of and the 2-by-2 payoff matrix is
[TABLE]
with cost-benefit ratio determining the relative outcomes in the payoff matrix. That is, if players and in a pairing both play , then both receive the payoff from this pairing; if player plays and player plays , then player gets payoff [math] and player gets payoff ; and so on.
Each time step of our microsimulation proceeds as follows. We uniformly at random pick one of the edges that connects a pair of players with different strategies, i.e., a link, denoted by . (In Section 5, we study a variant of the model where both and links are considered.) With probability (specified as a parameter), the two nodes and connected by edge consider updating their strategies; otherwise (i.e., with probability ), the edge is rewired. When a node reassesses its strategy along the edge , the node has probability specified by the Fermi function to change its state, as first proposed by szabo1998evolutionary , as specified in detail below. When link is rewired, the player with end state unilaterally drops the partnership with its neighbor with end state on the edge and picks (uniformly at random) another player from the remaining population outside its immediate neighborhood as its new partner. The left panel of Fig. 1 illustrates this rewiring process.
When a strategy updating event occurs, nodes and each consider their play across all of their neighbors, observing their total payoffs and . Then the strategy of node replaces that of along the edge —that is, copies ’s strategy—with probability given by the Fermi function
[TABLE]
where modifies the intensity of the bias towards the strategy with the higher payoff fudenberg2006evolutionary ; traulsen2007stochastic . Otherwise [that is, with probability ], copies . The value of can be interpreted here as representing the amplitude of noise in the strategy updating process vukov2006cooperation ; ren2007randomness ; that is, ignores the payoffs while yields deterministic imitation of the node receiving the higher payoff. The right panel of Fig. 1 illustrates the strategy updating process.
This partner switching evolutionary game stops when there are no discordant edges remaining in the network. That is, only and edges exist in the final state of the system, with the final network fissioning these two groups (cooperative and defective) into different components. In Fig. 2, we visualize a simulated network shortly before fission occurs, showing strong grouping of nodes by strategies.
We note that recent work khoo2016coevolution studied a spatially-embedded extension of the partner switching model in fu2009partner , introducing a range of rewiring distance and finding that a preference for global partner switching can coevolve with cooperation. However, in the present contribution we consider only random rewiring partner switching.
3 Semi-analytical methods of approximation
We study this generalized coevolving network model with simulations and approximate model equations. The frameworks of Mean Field theory (MF), Pair Approximation equations (PA) and Approximate Master Equations (AME) have all been used effectively in similar settings. The PA equations for these dynamics were obtained by fu2009partner . Among these three levels of approximation, AME can often be used to achieve the greatest accuracy gleeson2013binary , motivating us to develop the AME system here.
For comparison with the AME system that we will specify, we first look at the PA equations derived in fu2009partner , describing the dynamics of the quantities and —where —that count the numbers of nodes and edges corresponding to the two node states. For example, denotes the number of cooperators and denotes the number of links in the network. While the node states and network topology coevolve, the total numbers of nodes () and edges () remain conserved by the specified dynamics, requiring and . The evolution of these counts is then obtained through a moment closure approximating triple counts —where —in terms of the edge counts , and . Specifically, is approximated by the product of links and the average number of links that an node has, i.e. , hence . The resulting PA equations fu2009partner are as follows:
[TABLE]
where is the probability of strategy updating (versus edge rewiring), is the average utility of nodes, is the average utility of nodes, , , and controls the intensity of selection.
The AME method has been successfully used to approximate various processes on static and coevolving networks. The difficulty in deriving the AME system for the dynamics studied here is that the present transition probabilities depend on more than nearest neighbor states. For comparison, in the SIS model the recovery rate of an infected (I) node and the probability of infection along a given SI edge are both specified as constant parameters. Similarly, in typical voter model dynamics, the probability for a node to change its state relies on its neighbors’ opinions. In contrast, in the partner switching evolutionary game studied here, the local information impacting the decision of a node to change its state includes its total utility and that of its neighbors, which depend on the states of the neighbors’ neighbors. Since the binary-state AME framework only includes the node states, degrees, and neighbor states, we need estimate neighbors’ utilities in terms of these limited quantities. We will explain how we approximate these utilities after we introduce the equations below.
Let and , respectively, be the number of cooperating and defecting nodes of degree with defecting neighbors at time . We note that the numbers of nodes using each strategy are then given by the zeroth moments of the and distributions, and . Meanwhile, the first moments give the numbers of edges of each type: , and . Importantly, while the node states and network topology coevolve, the total numbers of nodes and edges remain conserved by the specified dynamics, requiring and .
The AME system of ordinary differential equations governing the and compartments is of course more complicated than the (relatively) compact PA equations (1). In the following paragraphs, we describe each of the terms that appear in this AME system:
[TABLE]
[TABLE]
To describe the AME derivation, we focus on explaining the equation, as the effects of corresponding terms of the equation are similar. The first three lines of equation (2), pre-multiplied by , are the effect of strategy updating, while the last line with are the effect of rewiring. To write the AME system, we track all counts flowing in and out of “center” class, . As visualized in Fig. 3, there are six such flows for the compartment, yielding the first six terms in the equation. The first term describes the rate at which nodes change to through comparing its utility with one of its neighbors, with the Fermi function evaluation as estimated below. Similarly, the second term captures nodes changing strategy to become after comparing their utility with their neighbors, with Fermi function evaluation also estimated below.
To estimate these Fermi function transition probabilities, we can directly compute the utility of the center node, since we know the numbers and types of its neighbors; but we need to estimate neighbors’ utilities in terms of the limited information available in the AME framework. Recall that the payoff matrix of our Prisoner’s dilemma game is
[TABLE]
where . For transitions by a strategy update out of the class, the center node is of state , with neighbors, of which are of state . Hence we can compute the center’s utility as
[TABLE]
Since we do not have any information about the neighbors of the neighbors of these nodes, we approximate utility by estimating the numbers and types of the neighbors of a neighbor. Specifically, we denote the expected number of neighbors of this neighbor by , and the expected number of neighbors of a neighbor by , where and will be estimated below and the accounts for the node itself. The estimate of the utility of the neighbor thus becomes
[TABLE]
and we compare the center node ’s utility with the estimate from one of its neighbors via the Fermi function, so that the estimated probability of the node changing its state in this consideration is
[TABLE]
Meanwhile, transitions by a strategy update out of the rely on the utility of a node,
[TABLE]
and estimating the utility of one of the neighbors via the numbers and types of its neighbors. We denote the expected number of neighbors of this neighbor (including the node in question) by + 1, and the number of neighbors of this neighbor as , both of which will be estimated below. With this notation, the expected utility of one of the neighbors is
[TABLE]
and the Fermi function estimate for the probability of the node changing its state in this comparison becomes
[TABLE]
The third to sixth terms in the equation are the effects of one of the neighbors changing its state and thus leading to a change of the total quantity of . Similar to the above, in order to make estimates of the transition probabilities we need a variety of estimates of the numbers and types of the neighbors’ neighbors, which we denote as follows. We use to provide the expected number of neighbors of a edge, as
[TABLE]
Similarly, denotes the expected number of neighbors of the node in a edge by
[TABLE]
Meanwhile, and estimate the numbers of neighbors of the node of a edge or of a edge, respectively:
[TABLE]
[TABLE]
Similarly, and provide the expected numbers of neighbors of the node of a edge or of a edge, respectively, while and give the numbers of neighbors of a edge or the node of a edge, respectively:
[TABLE]
[TABLE]
[TABLE]
[TABLE]
To compute the corresponding transition probability in each term, let us first consider the Fermi function factors on the second line in equation (2) specified by
[TABLE]
where
[TABLE]
and
[TABLE]
That is, this transition probability considers a neighbor of the center class, and we suppose this neighbor has neighbors in state (including the center node) and in state . Then we estimate this node’s utility and compare it with its typical neighbor (that is, the average utility ).
We note for emphasis here that other choices could have been made in this estimate; specifically, we explicitly know that one of the neighbors of this node is the center node. An alternative formulation of the AME system could be formed that separates out the state change of the neighbor in comparison with the center (thus removing the +1 contribution) from its state changes through interaction with other nodes. In order to keep the equations just a little simpler, and because of the good approximation we observe in our results below, we continue to employ the simpler estimate here.
We similarly estimate the Fermi function factors on the third line in equation (2) as
[TABLE]
where
[TABLE]
and
[TABLE]
In analogy to the above argument, we consider a neighbor of the center class and suppose this node has neighbors in state (including the center node) and in state . We estimate this node’s utility and compare it with the average utility of a node, .
The last line in the equation (2) describes the rewiring effect. The class flows to when the center node drops one of its defecting neighbors and rewires randomly to another . Similarly, the count decreases when the center node drops one of its defecting neighbors and rewires randomly to another . If the rewiring instead links to another node, these counts do not change. Finally, a node becomes if it is the recipient of a rewired edge (from another node), while the count decreases if a receives a rewired edge. The first group of square-bracketed terms on this line corresponds to the center node actively rewiring to some other node in the network while the second group in square brackets describes the center node passively receiving a rewired edge through the action of some other node in the network. Fig. 4 illustrates these active and passive rewiring processes.
Shifting our focus to the six flows in and out of the set depicted in the center of the lower row of Fig. 3, we similarly obtain equation (3) for the compartment. The resulting system contains coupled differential equations, where is the maximum degree a node can have in the network. Usually, the necessary value of for good accuracy depends on network structure and the mean degree. Here we choose . We numerically solve these differential equations to semi-analytically approximate the evolution of the system, using the ode45 solver in MATLAB until the solutions reach steady state.
While we consider PA and AME approaches here, we note that zhou2013link used an approach for studying adaptive SIS networks that treated the links as the objects, classifying them according to the disease states while tracking the degree and number of infected neighbors at both end nodes. This link-based method could generally improve the accuracy compared to AME; however, it also increases the number of coupled equations from to , where is the maximum degree in the network.
4 Simulations of the CD-switching model
To test the accuracy of the approximations, we study simulated dynamics on networks with nodes and edges. That is, the mean degree of the network is fixed to be . We set the parameter in the Fermi function used for imitating strategies. We study dynamics for different initial fractions of defectors, denoted by . We consider first, and explore the influence of this parameter later. We draw the initial networks from an Erdős-Rényi random graph model of nodes and edges distributed uniformly and independently between the nodes. For large , the initial degree distribution is approximated by the Poisson distribution of mean , with the probability of a selected node having degree being In the present model, updating only discordant edges (i.e., ), the dynamics stop when there are no discordant edges remaining. Unless stated otherwise, we perform simulations for each condition and report the average.
4.1 Final level of cooperation
To study the effect of the cost-benefit ratio and strategy update rate , we focus on the level of cooperation obtained in the final (stationary) states of the model, starting from an initial fraction of defectors . In Fig. 5, we present heat maps generated by simulations and AME approximations of the final fractions of cooperating nodes and CC links. As qualitatively expected, the final fraction of cooperating nodes is higher when the possible additional payoff for defecting, , is smaller. When is close to [math], we observe a moderate level of cooperators, as expected because the discordant edges can resolve themselves by rewiring with fewer strategy updates. Away from the small values of or , the final network states are dominated by defectors. The AME approximation generally captures these high and low cooperation regions of the parameter space, up to a modest displacement in the precise position of the phase transition.
In Fig. 6, we further explore the final level of cooperation and assess the accuracies of the two semi-analytical approximations. In the left panel of Fig. 6, we consider , , , and , plotting the final fraction of nodes in state versus the cost-benefit ratio, . There are three sets of outcomes here: simulations (markers), PA (dotted lines), and AME (dashed lines). For , there are no strategy updates and the final fraction of nodes is fixed at . For , strategy updates come into play with larger values of the cost-benefit ratio driving stronger incentive for nodes to defect, decreasing the final fraction of cooperators. As seen in the figure, the cases and include higher levels of cooperation at small that decrease slightly as increases before suddenly dropping to zero near and , respectively. This phenomenon is qualitatively captured by both PA and AME, though the quantitative description from AME is much more accurate, particularly in terms of these critical points. In contrast, the PA prediction of the fraction of cooperators drops to zero at much smaller values of . Similarly, in the case , the level of cooperation in the simulation results decreases with increasing , but without the sharp dropoff; in contrast to both the simulations and AME, the PA result drops to zero around .
Similarly, in the right panel of Fig. 6 we plot the final fraction of cooperators versus for different cost-benefit ratios, , , , and . For , no nodes update their strategies and the final fraction of nodes is . Larger increases both the number of strategy updates and their overall effect on the final state. In the cases and , is so small that there is almost no incentive to defect, so in the strategy updating consideration ’s and ’s are, holding everything else equal, almost indistinguishable. However, in rewiring an edge the node drops its neighbor, reducing the number of partners for , and over time the nodes experience lower payoffs because the numbers of their neighbors decrease. This decrease in degree results in lower total payoff, increasing the probability of a node changing to a strategy, leading to the dominance of ’s for small .
In contrast, for , the nodes become more disadvantaged with increased since the payoff of defecting is high, and the final fraction of cooperators decreases to zero around . The case is intermediate between the competing effects of these two extremes. The final fraction of cooperators increases for values of that are not too large due to the above-described rewiring effect. But sufficiently large strategy update rates (that is, fewer rewiring steps) combined with an apparently sufficient incentive to defect leads to final network states dominated by defectors. In the simulations, this transition for appears to occur near . While the PA and AME results qualitatively describe all of these effects well, AME does a better job capturing this transition point. Taking these results together, we note that in general the increased frequency of breaking and rewiring bonds (i.e., smaller ) can directly support cooperation, in agreement with szolnoki2008making ; szolnoki2009resolving , as observed here both in terms of the apparent phase transition and the behavior at large in the right panel of Fig. 6. But we also note that the results include situations where increasing this frequency (decreasing ) decreases the total level of cooperation, because of the complex interplay in the model dynamics.
4.2 Network dynamics
Having explored the populations in the final states, we consider how the networks evolve to those states. Specifically, we investigate the evolution of five fundamental quantities of the networks: the fractions of nodes in states and , and the fractions of edges that are , , and . We recall that the total numbers of nodes and edges are constant during the dynamics, requiring both fractions and fractions to be equal to . We visualize the average trajectories through this phase space for a collection of different parameters in Fig. 7, starting from an initial fraction of defectors distributed across an Erdős-Rényi random graph, comparing simulations with PA and AME predictions. As such, the initial fraction of , , and edges is , , and , respectively; that is, all networks start at , , . As the networks evolve, these fractions change, tracing out trajectories in the and coordinates in the figure. We perform 50 simulations at each set of parameter values and compute the averages across simulations at each time step. The networks evolve until there are no discordant edges (i.e. ).
For comparison, we include both PA and AME in the figure. While some features of the trajectories are better captured than others, both methods in general do reasonably well qualitatively, with better overall accuracy for AME, as seen in the figure. We note that both approximations are qualitatively incorrect for the case , . Recalling the simulation and AME results in the Fig. 5 phase diagram, we note that this parameter set is within the relatively narrow range between the observed location of the phase transition and the AME prediction.
4.3 Degree distributions
The AME method utilizes more information and has higher computational cost than PA, but as we have seen it generally provides a better approximation for the current model. Moreover, and unlike PA, AME explicitly includes information about degree distributions. In Fig. 8, we plot the final state degree distributions of and nodes, comparing the simulations and AME predictions. In each case, since we initialize with random strategies on the Erdős-Rényi random graph model, the initial degree distributions are approximately Poisson (for large ). We examine degree distributions in the final states, revisiting the same parameters considered previously: ; ; ; and . In the case , it is perhaps not surprisingly that AME gives an excellent prediction of the final degree distribution, since we observed excellent prediction of the trajectory of the dynamics in this case above (Fig. 7) . In the final state of this case, nodes dominate the whole network, with no nodes left.
In the other three cases, although the AME predictions are not as accurate, we can still see the AME provides a qualitative picture of the final degree distribution. Once again, the worst case of the four parameter sets visualized here is the , , where the AME prediction is on the wrong side of the phase transition.
4.4 The effect of the initial fraction of defectors,
In Fig. 9 we explore the effect of the initial fraction of defectors, . In this figure, we fix the parameter to be in the left panel and in the right panel, changing the other parameter ( and , respectively) to view the final fraction of cooperators against in stationary states, comparing simulation results (markers) with our AME approximation (lines). By construction, all results in these panels should connect the upper left and lower right corners because when (respectively, ) there are no () nodes, no discordant edges, and no dynamics.
In the left panel, for the different curves, the final fraction of cooperators first gradually decreases and then drops to zero suddenly at different values of . AME predicts this qualitative behavior for different levels of . Moreover, fixing , smaller results in higher cooperation, since the incentive of defecting is less when is small.
In the right panel, fixing , the diagonal line is the case , where there are no strategy updates and the cooperation level remains the same during the evolution. For small and moderate , the curves decrease with increasing (though more quickly for smaller ), followed by a sharp drop to zero at decreasing values of for increased . When is large (greater rates of strategy updates), the final cooperative level drops to zero for all non-zero . As we show in this panel, the AME curves suddenly drop to zero when .
5 A model variant with rewiring
In this Section, we study a slightly different variant of the partner switching evolution game model, also introduced by fu2009partner , where edges can also rewire as defector nodes seek out new partners that they might better exploit. We are particularly interested in how this additional rewiring affects the final cooperation level. In this variant, each time step starts by uniformly at random selecting from the union of and edges. (Recall that only edges were selected in the original model above.) If a edge is picked, the dynamics proceed as in the original model: with probability , the strategy update process happens; otherwise (that is, with probability ) the node along this edge unilaterally drops the partnership with its neighbor and rewires to another node. In contrast, if a edge is picked, then with probability nothing happens (there is no strategy update); otherwise (that is, with probability ) one of the two ends, selected with equal probability, drops the connection and rewires uniformly at random to another node in the network to whom it is not already connected. We note that the total numbers of nodes, , and edges, , remain constant, as in the original model.
This variant model stops evolving when there are no or edges remaining in the system; that is, all edges in a final, frozen state are . However, this does not require that all nodes are removed from the system, only that they are each left isolated with no connections. Another possibility, in which the model variant as stated above never reaches a stopping condition, is that the system evolves to a statistically stationary state where all nodes have been removed so that all edges are edges and the continued rewiring of these edges simply serves to repeatedly randomize the network of defector nodes. Since in this case there are no nodes left to emulate in a strategy update nor to connect to in a rewiring, we do not consider the dynamics further beyond this event. Both of these cases are visualized in Fig. 10. In the left panel, the nodes own all of the edges, having isolated the exploitation efforts of the nodes. In the right panel, only nodes remain and all edges are .
5.1 Semi-analytical methods of approximation
As before, we study this model variant with a combination of simulations and approximate analytic models, comparing the pair approximation (PA) and approximate master equation (AME) approaches with simulation results.
Following the notation of the original model in Sec. 3, the PA equations for this variant become.
[TABLE]
where we note the only difference between equation (4) and equation (1) is the appearance of the last term in the equation. This term captures the possibility in this variant model for a node to dismiss its defective partner and rewire to a node, thus decreasing the count.
Similarly, we modify the AME equations to account for the rewiring of edges, yielding
[TABLE]
[TABLE]
Again, the additional terms here compared to equations (2) and (3) account for the rewiring of edges, accounting for the rates of nodes abandoning neighbors and rewiring to nodes, nodes passively gaining new connections through this rewiring, and nodes being abandoned by their neighbors. Note the factors in the last two lines of the equation, accounting for an individual node on a keeping and losing the edge.
5.2 Final level of cooperation
As before, we compare the final fraction of cooperators in the simulations versus the PA and AME predictions. In the left panel of Fig. 11, we consider , , , and , plotting the fraction of nodes in the final states versus the cost-benefit ratio, . In the right panel, we fix values of and vary . Again, markers indicate simulation results, dotted lines are from the PA equations, and dashed lines are from AME.
Comparing Fig. 11 for this variant with the results in Fig. 6, the inclusion of rewiring here leads to slightly larger final fractions of nodes for small . In particular, whereas an existing link in the original model can only be changed if the state of one of the two nodes changes in a strategy update, the rewiring in this variant increases the numbers of edges and thus leaves open the possibility of nodes becoming more favorable because of their yet higher degree. However, for larger values of , the more rapid rate of strategy updating leads to defectors dominating the system before the cooperators can gain any advantage from increased degrees.
For further comparison, we consider the final fractions of nodes and edges for different combinations of and in Fig. 12, as compared with the similar plots in Fig. 5. While many of the general features remain the same in the presence of this DD-rewiring variant, a notable difference is in the final fraction of edges for small . In particular, at (no strategy updates), the original model results visualized in Fig. 5 limit to a final level of 0.75, since the initial edges (at 0.25 for ) cannot ever rewire under these settings. But these edges of course do rewire in the -rewiring variant, leading to limit to 1 as .
6 Conclusion
We have investigated a node-based Prisoner’s dilemma game played on a network coevolving with player strategy updates. Our study includes two model variants for rewiring: one where only edges can rewire, by nodes dropping a link to a neighbor in favor of a new partner; and a variant where edges can also rewire. We explore the parameter space to investigate the competing effects of strategy updates and partner switching, as well as the initial levels of cooperation versus defection. We compare our simulations to the existing pair approximation (PA) developed in fu2009partner , and we develop an approximation using approximate master equations (AME) to more accurately capture the transitions between the properties of the final states of these models. We also use the AME method to estimate the final-state degree distributions for different parameters.
We are particularly interested in the features that determine the final level of cooperation and overall utility in the network. Revisiting Fig. 6, and the corresponding results for the model variant in Fig. 11, it is clear that the most effective way to increase the final fraction of cooperators is to directly reduce the cost-benefit ratio, , so that players do not have as much incentive to defect. Alternatively, if the strategy update rate is small enough, the effects of rewiring give greater advantages to nodes as they accumulate larger numbers of playing partners, which then results in larger numbers of cooperators through the strategy updates. Of course, one can also directly decrease the initial fraction of defectors.
The question of how to maximize the total utility or payoff relates to the types of edges. In the partner-switching evolutionary game model we study here, the total number of nodes and edges remain fixed, no matter how the networks and node states change. Utility comes from every edge in the network, with each , , and edge contributing (from the two nodes) , , and , respectively, where . With no edges in the final state, maximizing the final overall utility reduces to maximizing the number of links. In particular, comparing the right panels of Figs. 5 and 12 for the two model variants, whether or not edges can rewire significantly affects the count (and thus the total system utility) for small strategy update rates.
Acknowledgments
Research reported in this publication was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health under Award Number R01HD075712. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. We are grateful to the reviewers for identifying multiple important references cited here.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1(1) Abramson, G. & Kuperman, M. (2001) Social games in a social network. Physical Review E , 63 (3), 030901.
- 2(2) Axelrod, R. (2006) The evolution of cooperation . Basic books, revised edition edition.
- 3(3) Bastian, M., Heymann, S., Jacomy, M. et al. (2009) Gephi: an open source software for exploring and manipulating networks.. ICWSM , 8 , 361–362.
- 4(4) Chen, Y., Qin, S.-M., Yu, L., Zhang, S. et al. (2008) Emergence of synchronization induced by the interplay between two prisoner’s dilemma games with volunteering in small-world networks. Physical Review E , 77 (3), 032103.
- 5(5) Dugatkin, L. A. & Mesterton-Gibbons, M. (1996) Cooperation among unrelated individuals: reciprocal altruism, by-product mutualism and group selection in fishes. Bio Systems , 37 (1), 19–30.
- 6(6) Ebel, H. & Bornholdt, S. (2002) Coevolutionary games on networks. Physical Review E , 66 (5), 056118.
- 7(7) Fletcher, J. A. & Zwick, M. (2007) The evolution of altruism: Game theory in multilevel selection and inclusive fitness. Journal of Theoretical Biology , 245 (1), 26–36.
- 8(8) Fowler, J. H. & Laver, M. (2008) A tournament of party decision rules. Journal of Conflict Resolution , 52 (1), 68–92.