Evolutionary games on cycles with strong selection

TL;DR

This paper analyzes fixation times in evolutionary games on cycle graphs under strong selection, revealing that intuitive expectations about fixation speed and demographic noise can be misleading depending on the update rule.

Contribution

It provides exact analytical results for fixation times under strong selection on cycle graphs, highlighting how different update rules affect demographic noise and fixation dynamics.

Findings

Fixation times can be polynomial in population size in certain coexistence games.

Demographic noise persists under strong selection with death-birth updates.

Strong selection can eliminate demographic noise with birth-death updates.

Abstract

Evolutionary games on graphs describe how strategic interactions and population structure determine evolutionary success, quantified by the probability that a single mutant takes over a population. Graph structures, compared to the well-mixed case, can act as amplifiers or suppressors of selection by increasing or decreasing the fixation probability of a beneficial mutant. Properties of the associated mean fixation times can be more intricate, especially when selection is strong. The intuition is that fixation of a beneficial mutant happens fast (in a dominance game), that fixation takes very long (in a coexistence game), and that strong selection eliminates demographic noise. Here we show that these intuitions can be misleading in structured populations. We analyze mean fixation times on the cycle graph under strong frequency-dependent selection for two different microscopic evolutionary update rules (death-birth and birth-death). We establish exact analytical results for fixation times under strong selection, and show that there are coexistence games in which fixation occurs in time polynomial in population size. Depending on the underlying game, we observe inherence of demographic noise even under strong selection, if the process is driven by random death before selection for birth of an offspring (death-birth update). In contrast, if selection for an offspring occurs before random removal (birth-death update), strong selection can remove demographic noise almost entirely.

Equations

$$\displaystyle\begin{array}[]{c|cc}&A&B\\ \hline\cr A&a&b\\ B&c&d\end{array}$$

$$\begin{array}{r}{\varphi }^i=\frac{1+{\sum }_{k=1}^{i-1}{\prod }_{l=1}^k\frac{T^{l-}}{T^{l+}}}{1+{\sum }_{k=1}^{N-1}{\prod }_{l=1}^k\frac{T^{l-}}{T^{l+}}}\end{array}$$

$$r^j=(1-T^{j+}-T^{j-})+T^{j+}{\varphi }^{j+1,j}+T^{j-}{\varphi }^{j-1,j}$$

$${\varphi }^{i,j}(r^j{)}^{t-1}(1-r^j)$$

$$t^{1,j}=\sum _{t=1}^{\infty }{\varphi }^{1,j}(r^j{)}^{t-1}(1-r^j)t$$

$$\begin{array}{r}{t}^1=\sum _{j=1}^{N-1}\frac{{\varphi }^{1,j}}{T^{j+}(1-{\varphi }^{j+1,j})+T^{j-}(1-{\varphi }^{j-1,j})}\end{array}$$

$${\tilde{t}}^{1j}=\frac{{\varphi }^j}{{\varphi }^1}{t}^{1,j}$$

$$t^{1\to N}=\sum _{j=1}^{N-1}\frac{{\varphi }^j}{{\varphi }^1}\frac{{\varphi }^{1,j}}{T^{j+}(1-{\varphi }^{j+1,j})+T^{j-}(1-{\varphi }^{j-1,j})}$$

$$T_{\text{wm}}^{i+}$$

$$T_{\text{wm}}^{i-}$$

$$\begin{array}{r}{T}_{\text{dB}}^{i+}=\{\begin{array}{ll}\frac{2}{N}\frac{1}{1+{\text{e}}^{-\beta (2b-2d)}}& \text{if}i=1\\ \frac{2}{N}\frac{1}{1+{\text{e}}^{-\beta (a+b-2d)}}& \text{if}1<i<N-2\\ \frac{2}{N}\frac{1}{1+{\text{e}}^{-\beta (a+b-c-d)}}& \text{if}i=N-2\\ \frac{1}{N}& \text{if}i=N-1\\ 0& \text{if}i=0,N\end{array}\end{array}$$

$$\begin{array}{r}{T}_{\text{dB}}^{i-}=\{\begin{array}{ll}\frac{1}{N}& \text{if}i=1\\ \frac{2}{N}\frac{1}{1+{\text{e}}^{+\beta (a+b-c-d)}}& \text{if}i=2\\ \frac{2}{N}\frac{1}{1+{\text{e}}^{+\beta (2a-c-d)}}& \text{if}2<i<N-1\\ \frac{2}{N}\frac{1}{1+{\text{e}}^{+\beta (2a-2c)}}& \text{if}i=N-1\\ 0& \text{if}i=0,N\end{array}\end{array}$$

$$T_{\text{Bd}}^{1+}=\frac{{\text{e}}^{\beta 2b}}{{\text{e}}^{\beta 2b}+2{\text{e}}^{\beta (c+d)}+(N-3){\text{e}}^{\beta 2d}}$$

$$T_{\text{Bd}}^{1-}=\frac{{\text{e}}^{\beta (c+d)}}{{\text{e}}^{\beta 2b}+2{\text{e}}^{\beta (c+d)}+(N-3){\text{e}}^{\beta 2d}}$$

$$\begin{array}{r}{T}_{\text{Bd}}^{i+}=\{\begin{array}{ll}\frac{{\text{e}}^{\beta (a+b)}}{F_i}& \text{if}1<i<N-1\\ \frac{{\text{e}}^{\beta (a+b)}}{F_{N-1}}& \text{if}i=N-1\\ 0& \text{if}i=0,N\end{array}\end{array}$$

$$\begin{array}{r}{T}_{\text{Bd}}^{i-}=\{\begin{array}{ll}\frac{{\text{e}}^{\beta (c+d)}}{F_i}& \text{if}1<i<N-1\\ \frac{{\text{e}}^{\beta 2c}}{F_{N-1}}& \text{if}i=N-1\\ 0& \text{if}i=0,N\end{array}\end{array}$$

$$\begin{array}{r}{\varphi }_{\text{dB}}^i={\varphi }_{\text{Bd}}^i=\frac{i}{N}\end{array}$$

$$t_{\text{dB}}^1=t_{\text{Bd}}^1$$

$$t_{\text{dB}}^{1\to N}=t_{\text{Bd}}^{1\to N}$$

$$t_{\text{dB}}^{1\to N}=t_{\text{dB}}^1$$

$$t_{\text{Bd}}^{1\to N}=t_{\text{Bd}}^1$$

$$T_{\text{dB}}^{1+}$$

$$T_{\text{dB}}^{i+}$$

$$T_{\text{dB}}^{(N-2)+}$$

$$T_{\text{dB}}^{(N-1)+}$$

$$T_{\text{dB}}^{1-}$$

$$T_{\text{dB}}^{2-}$$

$$T_{\text{dB}}^{i-}$$

$$T_{\text{dB}}^{(N-1)-}$$

$$T_{\text{Bd}}^{1+}$$

$$T_{\text{Bd}}^{i+}$$

$$T_{\text{Bd}}^{(N-1)+}$$

$$T_{\text{Bd}}^{1-}$$

$$T_{\text{Bd}}^{i-}$$

$$T_{\text{Bd}}^{(N-1)-}$$

$$T_{\text{wm}}^{i+}$$

Full text

I Introduction

Evolutionary game theory models Darwinian selection among genetically hard-wired strategic traits or behaviors in a population Maynard Smith and Price (1973); Hofbauer and Sigmund (1988). Often the interaction between behaviors is cast into an evolutionary game, and the performance in this evolutionary game determines the rate at which strategies spread. As payoffs from the game are mapped to fitness, i.e. the expected number of offspring in the near future, more successful behaviors have a higher tendency to spread in the population. In infinitely large populations this spreading of successful behaviors due to Darwinian selection is described by the deterministic replicator dynamics Taylor and Jonker (1978); Hofbauer et al. (1979); Zeeman (1980); Ohtsuki and Nowak (2006a); Ohtsuki and Nowak (2008). In finite populations fluctuations cannot be neglected and the evolutionary dynamics become stochastic Fogel et al. (1998); Nowak et al. (2004); Traulsen et al. (2005); Hilbe (2011); Black and McKane (2012). A parameter that governs the interplay between the determinism of selection and intrinsic stochasticity in finite populations is the strength of selection Altrock et al. (2010). Neutral evolution emerges in absence of selective differences. If selection acts, the stochastic evolutionary dynamics become payoff dependent, which can be the same in each state (constant selection) or entirely state dependent (frequency-dependent selection), whereby the state is defined as the number of mutants. In the case of frequency-dependent selection the probability that one strategy replaces another can be fairly complicated. In particular, structure of the population itself influences the evolutionary game and the potential success of strategic behaviors Lieberman et al. (2005); Broom and Rychtář (2008); Broom et al. (2010); Hindersin and Traulsen (2014); Kaveh et al. (2015).

In stochastic evolutionary game dynamics the event of interest is fixation of a mutant Nowak (2006). Two quantities have been of special interest: the fixation probability and the expected fixation time Maruyama (1970); Nei and Roychoudhury (1973); Slatkin (1981); Taylor et al. (2006); Altrock and Traulsen (2009); Altrock et al. (2010). Fixation probabilities have served as the gauge whether a graph can be an amplifier of suppressor of selection Lieberman et al. (2005); Hindersin and Traulsen (2015). An open problem is the general quantification of fixation times in graph structured populations. Fixation times quantify the expected time new traits need to take over the population. For constant selection, recent findings have established that evolution can slow down substantially in populations where selection is amplified Frean et al. (2013), and that there are no obvious relations between fixation probabilities and fixation times on graphs Hindersin and Traulsen (2014). In addition, structured population dynamics may be different if selection occurs before or after random death of individuals Ohtsuki et al. (2006); Hindersin and Traulsen (2015). Here we seek to start closing this gap using analytical procedures in an evolutionary game between resident strategy and a mutant strategy. To this end, we consider dynamics on the simplest structure, in which exactly one individual occupies a node on an undirected cycle graph Ohtsuki and Nowak (2006b), and focus on strong selection Altrock et al. (2010).

This manuscript is organized in the following way. First, we introduce the stochastic evolutionary dynamics. We review the well-mixed population and discuss the analytical expressions for fixation probabilities, sojourn times and fixation times. Then, we introduce the transition probabilities of birth-death and death-birth updating on cycle graphs. In the results section, we consider neutral evolution and briefly address constant selection before we turn to strong frequency dependent selection. Then, we discuss standard cases of two player-two strategy games between a mutant strategy $A$ and a resident strategy $B$, given by the payoff matrix

[TABLE]

We consider strategic dominance games of the mutant strategy in which $A$ always has a higher payoff ($a>c$, $b>d$), coordination games in which both mutants and residents receive highest payoffs when interacting with their own types ($a>c$, $b<d$), and coexistence games in which both mutants and residents receive highest payoffs when interacting with the other respective strategy ($a<c$, $b>d$). As particular examples, we discuss the Prisoner’s Dilemma (where defection dominates cooperation) an the and the Snowdrift Game (where defection and cooperation can coexists).

II Evolutionary game dynamics

First we describe the discrete-time Markov chain model resulting from subsequent birth and death events in a population of finite fixed size. Two key assumptions are that we start with a single mutant individual and that no further mutations occur. Thus, on a cycle graph, the mutant population can only grow as a cluster. We also make the assumption that replacement graph and interaction graph are identical (Ohtsuki and Nowak, 2006b). The resulting Markov chain eventually gets absorbed either of the boundary state of no or all mutants. The population size is $N$ and we denote the evolutionary transition probabilities by $T^{i\pm}$. Here, $i$ is the number of mutant individuals that at any time can increase or decrease by exactly one. The process stays in state $i$ with probability $1-T^{i+}-T^{i-}$.

We can then examine the fixation probability of a group of $i$ type $A$ individuals, $\phi^{i}$, as well as other quantities of interest without specifying the transition probabilities. The fixation probability follows from solving the backward Kolmogorov equation $\phi^{i}=T^{i+}\phi^{i+1}+T^{i-}\phi^{i-1}+(1-T^{i+}-T^{i-})\phi^{i}$ recursively (see e.g. Karlin and Taylor, 1975; Nowak, 2006; Altrock et al., 2010) and is given by

[TABLE]

where only the ratios of transition probabilities enter.

To characterize the expected time scale of the evolutionary process, one can consider two different quantities. First, the mean unconditional fixation time describes the expected time the process takes to reach either extinction or fixation of the mutant, which occurs with probability one. Second, the mean conditional fixation time describes the expected number of time steps the process takes to fixation of the mutant, which occurs with probability $\phi^{1}$. One way to derive an expression for mean fixation times is to think about the expected time spent in each intermediate state (including waiting times) $j=1,2,\dots,N\!-\!2,N\!-\!1$, which are called sojourn times. The sojourn time of a particular state $j$ can be found by considering its escape process. Say that at time $t^{0}$, the process is in state $j$. Then, the probability that it either stays or ever returns to that state $j$ (denoted as the super-script) is given by

[TABLE]

Here, $\phi^{k,l}$ is the probability to start in state $k$ and ever return to state $l$ ($\phi^{k,k}=1$) (Altrock et al., 2012), which is not conditioned on fixation. The conditional probability to start in state $i$, return a positive number of time steps $t$ to that state, but then escape is given by

[TABLE]

The first factor of Eq. (7) describes the probability to ever get from $i$ to $j$, the second factor describes recurrence such that the total time spent amounts to exactly $t$ time steps, and the third factor describes definite escape from state $j$.

The mean sojourn time in state $j$, starting from one mutant, $i=1$, is thus given by the first moment of this conditional probability

[TABLE]

This geometric sum can be solved exactly, and inserting the definition of $r^{j}$ we obtain an exact expression for the mean sojourn time of state $j$, $t^{1,j}=\phi^{1,j}/(1-r^{j})$. The mean unconditional fixation time is then given by the sum over all sojourn times (Ewens, 2004; Ohtsuki and Nowak, 2006b; Altrock et al., 2012)

[TABLE]

The mean conditional fixation time can be found in a similar way, only resorting to conditional transition probabilities of the Markov process $\widetilde{T}^{i\to j}=\left(\phi^{j}/\phi^{i}\right)T^{i\to j}$ (Ewens, 2004), from which we find the probabilities to start in $i$ and ever visit $j$ under the condition of fixation in $N$, $\widetilde{\phi}^{i,j}=\left(\phi^{j}/\phi^{i}\right)\times\phi^{i,j}$. Consequently, the mean sojourn time in state $j$ conditioned on fixation, starting from a single mutant, can be expressed by the mean unconditional sojourn time

[TABLE]

such that the mean conditional fixation time can be calculated by

[TABLE]

whereby the stationary probabilities to ever go from state $i$ to state $j$, $\phi^{i,j}$ are derived in the book by Ewens Ewens (2004). We repeat them in our Appendix, Eqs. (97) and (98).

II.1 Well-mixed population

The reference case for evolutionary game dynamics is the well-mixed population Nowak and Sigmund (2004). In the well-mixed population, an expected payoff is calculated taking into account interactions between all individuals. This is equivalent to a fully connected unweighted graph. Here, we briefly recall the properties of the Moran process of frequency dependent selection in well-mixed populations. Formally, the Moran process is introduced as a birth-death process, but in a well-mixed population, the ordering of a fitness-proportional birth and a random death event does not have any influence on the dynamics as long as we include self replacement, which is commonly assumed Hindersin and Traulsen (2015).

The well-mixed population assumes that there are interactions between all individuals, which lead to an average payoff, and in turn determines selection via a specific choice of payoff to fitness mapping (Wu et al., 2010, 2013, 2015). In this paper, we focus on an exponential payoff to fitness mapping (Traulsen et al., 2008). If $i$ and $N-i$ are the numbers of $A$ and $B$ individuals, the expected payoff of any $A$ individual is given by $\pi_{A}=a\,(i-1)/(N-1)+b\,(N-i)/(N-1)$. The expected payoff of any $B$ individual is given by $\pi_{B}=c\,i/(N-1)+d\,(N-i-1)/(N-1)$. Then, the Moran process is a Markov chain with transition probabilities to neighboring states given by

[TABLE]

These transition probabilities are non-linear functions of the number of mutant individuals. For most types of game in well-mixed populations, the states between all-mutant and all-resident have different probabilities to increase or decrease the number of mutants.

There are two popular mechanisms often used to describe evolutionary dynamics on graph structured populations (Lieberman et al., 2005; Antal et al., 2006; Ohtsuki et al., 2006; Szabó and Fáth, 2007; Zukewich et al., 2013). First, in the death-birth process, there is random death of an individual and subsequent selection among its neighbors for filling the vacant spot. Thus, competition is only among individuals of the immediate neighborhood of the vacant spot. Second, in the Birth-death process, there is selection for birth of an identical offspring within the entire population, before random death of a neighbor of the reproducing individual occurs. Hence, the death-birth process models random death which precedes local competition, whereas the birth-death process models global completion which precedes random death. The basic difference between the two update mechanism on the cycle are depicted in Figure 1, and described in more detail in the following.

II.2 Death-birth process on a cycle

The death-birth update on a cycle works as follows. Each node of the graph represents one individual. First, all individuals play the evolutionary game with their two neighbors. Then a random individual is selected for removal. The neighbors of this empty spot then compete for placing an identical offspring. The number of mutants only changes when random death occurs at the boundary, as mutants grow in a cluster. There are two sites that can be chosen for random death at the boundary of residents and mutants, which occurs with probability $2/N$. Competition among the two neighbors of the vacant spot then leads to a probability that a mutant fills the spot. Let $f_{A}$ and $f_{B}$ be the fitness values of the two neighbors of the vacant spot. Then, the probability that the $A$ individual’s offspring occupies the vacant spot is $f_{A}/(f_{A}+f_{B})=1/(1+f_{B}/f_{A})$, resulting in the transition probabilities

[TABLE]

and

[TABLE]

The states with one mutant individual or one resident individual are special in the sense that there is no competition, only random selection for removing that last individual. Interestingly, for $2<i<N-2$, the expansion of a cluster of $A$ players does not depend on the payoff parameter $c$. This makes sense as the only individual affected by $c$ had to be subject to random death before selection. Similarly, the expansion of a $B$-cluster does not involve the payoff parameter $b$. Note also that since all transition probabilities are determined by payoff differences, this process is invariant under adding a constant to each payoff value, and multiplication of the payoff matrix with a positive real number changes the strength of selection.

II.3 Birth-death process on a cycle

For the frequency dependent birth-death (Bd) update on a cycle, with exponential payoff to fitness mapping, we find more complicated transition probabilities. The transition from one mutant to a cluster of two mutants occurs with probability

[TABLE]

and the extinction of a single mutant occurs with probability

[TABLE]

In similar fashion we find

[TABLE]

and

[TABLE]

where we used the abbreviations $F_{i}$ for the total fitness of the population in state $i$, see Appendix A. Again, the birth-death process is invariant under adding a constant to the payoff values. In this case, there are only three states for which the ratio of transition probabilities $t_{\text{Bd}}^{i-}/t_{\text{Bd}}^{i+}$ deviates from the constant value $e^{-\beta(a+b-c-d)}$, we obtain a different value only for $i=1,2,N-2,$ and $N-1$.

III Results

III.1 Neutral evolution

Neutral evolution means that the probability to increase or to decrease the number of mutants on the cycle are always the same, irrespective of the number of mutant individuals. On a cycle, this probability is the same for any number of mutants as long as neither mutant or resident is extinct. The probably to increase or decrease the number of mutants also becomes independent of the specific update mechanism used and simply amounts to $1/N$. Under any of the two update rules we thus find

[TABLE]

for the fixation probability starting from $i$ mutants. For the fixation times stating from one mutant we obtain

[TABLE]

for the mean unconditional fixation times, and

[TABLE]

for the mean conditional fixation times. For the well-mixed population the the mean unconditional fixation time of the neutral process is $t^{1}_{\text{wm}}=N\,H_{N-1}$, where $H_{k}=1+1/2+\dots+1/k$ is the harmonic number, which increases logarithmically for large $k$. Thus, fixation or loss is slower on the cycle graph, where it scales as $\sim N^{2}$. Also the mean conditional fixation time on the cycle ($\sim N^{3}$) is much longer than in a well mixed population, where it only scales quadratically in $N$, $t^{1\to N}_{\text{wm}}=N(N-1)$. Often, the times under neutral evolution set the reference against which the mean times under selection are measured (Altrock et al., 2010, 2012; Hindersin and Traulsen, 2015).

III.2 Constant selection

For constant selection we assume $f_{A}=\text{e}^{\beta\,r}$ and $f_{B}=1$. In this case, the death-birth process and the birth-death process are inherently different (see Appendix). The death-birth process under strong constant selection in favor of the mutant, $\beta\,r\to\infty$, leads to mean sojourn times that are constant proportional to $N$. The birth-death process under strong constant selection leads to mean sojourn times in state $j$ that are equal to $j$. Ultimately, this leads to

[TABLE]

where the conditional and unconditional fixation times are identical because the fixation probability quickly converges to 1. We can also make a quick comparison of the Moran process on the ring (the Bd process) and the Moran process in a well-mixed population. In Appendix D calculate approximations for these two strongly related processes for finite but large $r$, resulting in the mean conditional fixation times $t^{1\to N}_{\text{Bd}}\approx N(N-1)/2+N(N-2)\text{e}^{-\beta\,r}$ and $t^{1\to N}_{\text{WM}}\approx N\,H_{N-1}+2\,N\,H_{N-2}\,\text{e}^{-\beta\,r}$. These relations are able to describe how the mean fixation times approach the value of the strong selection limit value for any population larger than $N=2$.

III.3 Strong frequency-dependent selection

Now, we discuss the semi-analytical solutions given by Equations (9) and (11), for frequency-dependent selection. We focus on the limiting case of strong selection, $\beta\to\infty$, to obtain further analytical insights. These insights can guide our intuition as to how strong frequency dependent selection and spatial structure impact times to extinction or fixation in structured populations of finite size.

We now quantify how the different update rules on the cycle behave in the strong selection limit in order to develop an intuition for the limiting behavior of fixation probabilities, as well as mean fixation times. We focus on non-trivial sets in payoff-space, e.g. $a+b>c+d$, and exclude special cases which are of measure zero in parameter space, e.g. $b=d$. We start with the dB update on the cycle. The limiting cases of $\beta\to\infty$ for the transition probabilities to increase the number of $A$ individuals are given by

[TABLE]

In the same way, we obtain the limiting cases of the transition probabilities to decrease the number of $A$ individuals:

[TABLE]

where again, the last step before $B$ fixes in the population is independent of the game.

The spread of a single $A$ mutant is impossible under strong selection if the payoff of $A$ against $B$ is lower than the payoff of $B$ against itself. However, the loss of the single $A$ mutant occurs with an expected waiting time proportional to the size of the population. Fixation of $A$ can only occur under strong selection if this initial step is possible ($b>d$) and if additionally $a+b>2d$, as well as $a+b>c+d$.

If there is an unstable mixed Nash equilibrium in the game (which is then a coordination game), fixation of $A$ is only possible in a subset of all games. Generally, under the dB process fixation and extinction may still take long in large populations, as the non-vanishing transition probabilities are proportional to $N^{-1}$.

In contrast, the transitions rates of the Bd process (evolutionary Moran process on graphs (Ohtsuki et al., 2006)) can become independent of the population size, The limiting cases for the transition probabilities are given by the following conditions, where it is important to note that for non-trivial transition probabilities, all the payoff relations have to be fulfilled,

[TABLE]

and

[TABLE]

Hence, the last transition before extinction or fixation of $A$ can become entirely deterministic. All other transitions have a non-trivial limiting case of $1/2$, independent of the size of the population, which is due to the fact that the selection (birth) step comes first, which deterministically selects the best performing individual(s). This selection step is then followed by the death step, which can only select one of the two neighbors of the parent. Thus, under strong selection, fixation in a Bd process can occur at a much faster rate than fixation in a dB process. In addition, it is possible that strong selection on the cycle becomes static; neither increase or decrease of the mutant strategy $A$ are possible.

For comparison we again consult the well-mixed population, where, based on Eqs. (12) and (13), the dynamics always depends on the payoff in the form of relations between expected payoffs $\pi_{A}$, $\pi_{B}$. As a consequence, the strong selection limiting cases in the well-mixed population are

[TABLE]

The dynamic time scales of fixation or extinction processes are generally population size dependent, but depend on the payoff matrix only implicitly. For sufficiently large but finite population size $N$, the mean unconditional fixation time of a strategic dominance game or a suitable coordination game is proportional to $N\,\text{log}[N-1]$ plus a constant (in fact Euler’s constant Graham et al. (1994)), but generally diverges exponentially for coexistence games Traulsen et al. (2006); Altrock et al. (2012).

IV Discussion

IV.1 Strategic dominance

We speak of strategic dominance if in any one shot interaction, $A$ does always better than $B$, i.e. $a>c$, $b>d$. In the well mixed population, this immediately leads to a relation for the transition rates for any strength of selection, $T_{\text{wm}}^{i+}\geq T_{\text{wm}}^{i-}$. In structured populations, additional conditions on the payoffs may be required for this to be fulfilled.

For the death-birth process the strong selection limiting case leads to the following payoff relations. If the three inequalities $a+b>2d$, $a+b>c+d$, and $2a>c+d$ hold, we obtain limiting cases of the transition rates and sojourn times,

[TABLE]

A single mutant has an extinction probability of only one-third, but a probability to reach fixation of two-thirds: once the mutation spreads to two or more individuals, it is bound to take over with certainty. The fixation times can again be calculated by summation over the sojourn times, using Eqs. (9) and (11), which leads to

[TABLE]

and

[TABLE]

In both cases, the leading order term is quadratic in the population size $N$.

The second case of non-vanishing fixation probability of a single mutant is when $2a<c+d$. Then, fixation of the dominant mutant takes considerably longer on average but the fixation probability of a single mutant is still $2/3$:

[TABLE]

such that the mean fixation times become

[TABLE]

Hence for large population size, conditional fixation takes on average 50% longer than unconditional fixation.

Mutant-fixation with non-vanishing probability in the birth-death process requires the four conditions $2b>c+d$, $a+b>2d$, $a<b$, and $a+b>2c$. As a result, we find the following transition probabilities and sojourn times

[TABLE]

The mutant fixes with probability one. We obtain fixation times linear in population size

[TABLE]

We can also find generic payoff matrices in which the mutant neither spreads nor goes extinct, despite strategic dominance of one strategy.

IV.2 Coordination games

In coordination games, the interactions between two individuals of the same type always yield higher payoffs than interactions between two different types, hence the payoff relations for all coordination games are $a>c$ and $b<d$. In a well-mixed population, $A$ can then not invade $B$ and vice versa. Coordination games are used to study evolution of technological standards (Cooper, 1998), emerge in a subset of strategies in collective risk dilemmas (Milinski et al., 2008; Hilbe et al., 2013), and have a population genetic equivalent in genetic underdominance (heterozygote disadvantage) between two alleles of the same gene (Altrock et al., 2011).

In the death-birth process, the limiting cases of the transition matrix (25)-(28) immediately tell us that a mutant $A$ cannot invade, and we obtain

[TABLE]

Thus, the conditional mean fixation time is not a meaningful quantity to compute. The unconditional mean fixation time, which only measures extinction of $A$, follows as

[TABLE]

This asymptotic relation holds for all coordination games under strong selection.

The birth-death process leads to limiting cases in which the single mutant cannot invade. If the inequalities $c>d$ and $d+c>2b$ hold, we discover the following transition probabilities and sojourn times

[TABLE]

The mutant goes extinct with probability one in two time steps on average,

[TABLE]

IV.3 Coexistence

In coexistence games, it is always better for a individual to interact with an individual that plays a different strategy. The payoff relations are $a<c$ and $b>d$. Such strategic interactions emerge when cooperators generate a benefit that can be exploited by both cooperators and defectors (MacLean et al., 2010). A coexistence game was the original motivation to study a game in an evolutionary context, namely the Hawk-Dove game (Maynard Smith and Price, 1973), which allows a coexistence of such behaviors, in which common strategies outperform rare strategies (Doebeli and Hauert, 2005).

In order to observe non-vanishing fixation probability of a single $A$ mutant in the death-birth process, we require the following payoff-conditions to hold: $a+b>c+d$, $a+b>2d$, and $c+d>2a$ as an additional relation that determines the speed to fixation. Then, we can summarize the limiting cases of transition probabilities and sojourn times as follows

[TABLE]

In this case of coexistence game, the mean unconditional fixation time sums up to a cubic polynomial in $N$

[TABLE]

and the mean conditional fixation time amounts to

[TABLE]

If the coexistence game does not fulfill $c+d>2a$ we obtain

[TABLE]

which is slightly different than (45). Thus

[TABLE]

There are two generic regimes of payoffs which lead to significantly different fixation times in coexistence games. In the first parameter regime, the mutation can go extinct even after it has spread to intermediate frequencies, and the fixation times scale in leading order with $N^{3}$. In the second regime the times scale with $N^{2}$, because in (77), the process can go down from $i$ again, but not so in (84), which make the process more efficient.

The birth-death process under a coexistence game requires the following five relations between payoffs in order to lead to non-vanishing fixation probability of the mutant: $2b>c+d$, $a+b>c+d$, $a+b>2d$, $a<b$, and $a+b>2c$, which together result in $\phi^{1}\to 1$ and the same transition probabilities, sojourn times and fixation times we have already discovered for strategic dominance, Table (59). Note that, if we relaxed the condition $a+b>2c$, such that $T_{\text{Bd}}^{(N-1)+}\to 0$ and $T_{\text{Bd}}^{(N-1)-}\to 1$, fixation would take infinitely long to occur. Similarly, in coexistence games with $a+b<c+d$, we find $T_{\text{Bd}}^{(i>1)+}\to 0$ but $T_{\text{Bd}}^{1+}\to 1$, and at the same time $T_{\text{Bd}}^{1-}\to 0$ but $T_{\text{Bd}}^{(i>1)-}\to 1/2$, $T_{\text{Bd}}^{(N-1)-}\to 1$: the process gets trapped between the states 1 and 2, see Figure 2.

In coexistence games the intuition is that mean fixation times tend to infinity Traulsen et al. (2006). For both the death-birth and the birth death process on the cycle graph, we have shown that there are generic subsets of game parameters that allow fixation of the mutant with non-vanishing probability in a finite amount of time. Compared to a dominance game, the time scales in such coexistence games may be longer by a factor $N$ in the death-birth process, but identical for the birth-death process.

IV.4 Prisoner’s dilemma vs. snowdrift game

We now turn to two concrete examples of social dilemma situations. We denote cooperators as the resident type and defectors as the mutant type. First, a game between a cooperative resident strategy $C$ and a defective mutant strategy $D$ is the prisoners’ dilemma game, where cooperation corresponds to to offer a benefit to the co-player at a cost smaller than the benefit:

[TABLE]

The distribution of the conditional fixation times of this game, for both death-birth and birth-death update rules are shown in Figure 3. We assumed benefit$=8$ and cost$=5$ and measured the mean conditional fixation times for the death-birth and the birth-death processes. Due to the inherent stochasticity of the death-birth process even under very strong selection, the fixation time is considerably larger and more variable (compare (45) with (59)).

Second, in the snowdrift game there is a benefit to the cooperator even when facing a defector. Cooperation still comes at a cost, but this cost is shared under mutual cooperation. A respective example payoff matrix of this form of coexistence game is

[TABLE]

This payoff configuration leads to the cases (77) for the death-birth process, and to (59) for the birth-death process. Hence with Eq. (78) we expect long fixation times in the death-birth precess, and fast fixation in the birth-death process, Eq. (60). Our simulations show that fixation times in the death-birth process of this coexistence game tend to be extremely widely distributed, Figure 4, whereas the death-birth processes fixation times are narrowly distributed around the mean for strong selection. For comparison, in the neutral case, 4 (c), rare events of long fixation times are less common.

These examples show that while in birth-death driven evolutionary dynamics in structured populations an advantageous mutant can take over quickly under strong selection, the inherent stochasticity of the death-birth driven dynamics can lead to a wide distribution of fixation times. For coexistence games, birth-death processes can again lead to fast fixation in times of order $N$, and death-birth process fixes in times of order $N^{3}$, but with a very wide distribution. This leading order is much smaller than in the well-mixed population case, in which fixation times tend to infinity for strong selection Antal and Scheuring (2006); Altrock et al. (2012).

V Summary and Conclusions

Selective forces of stochastic evolutionary dynamics in structured populations are driven by the underlying update rule Ohtsuki et al. (2006); Frean et al. (2013); Hindersin and Traulsen (2015) and by how payoff is translated into fitness Maciejewski et al. (2014). Here we focus on strong selection on the cycle graph Ohtsuki and Nowak (2006b) and an exponential payoff to fitness mapping. Using both analytical calculations and simulations, we show that outcomes of the death-birth process may differ drastically from outcomes of the birth-death process. Under strong selection, transitions in the death-birth process remain stochastic (proportional to $1/N$), as the random death-step before selection always depends on population size. The maintenance of this degree of demographic noise is surprising, as for neutral evolution conditional fixation times are of leading order $N^{3}$, and there are games for which the strong selection fixation times are also of this leading order (or of leading order $N^{2}$). Under the birth-death process, fitness effects can eliminate dependence on population size, and transition rates become constant under strong selection. Hence, the leading order of mean fixation times is $N$.

For dominance games, both the death-birth and the birth death process can lead to non-vanishing fixation probability and finite fixation times when the payoff of mutants interacting with a resident, $b$, is sufficiently large compared to all other payoffs. This condition ensures that the mutant can invade and stays advantageous at the boundary between residents and mutants. For the example of a strategic dominance game, such as the the prisoner’s dilemma, not all payoff configurations allow mutant invasion Ohtsuki and Nowak (2006b): if the cost of cooperation is sufficiently low, a population of cooperators is immune to invasion and fixation of a defective mutant under both update processes. This result stands in contrast to the well-mixed population. If the payoff configuration is such that defective mutants can invade, i.e for high benefit and low cost, the speed of fixation is expected to take long and fixation remains highly stochastic for the death-birth process, even under strong selection. For a birth-death update driven process, rapid takeover by an advantageous mutant can be observed, see Figure 5.

Stochastic evolutionary dynamics of coordination games on the cycle graph show closest resemblance to their well-mixed counterpart: under strong selection, mutants cannot invade, and the respective extinction times are short (of order $N$ in the death-birth process, of order 1 in the birth-death process), see Figure 5. The snowdrift game promotes coexistence of cooperators and defectors in well-mixed populations under strong selection Altrock et al. (2010, 2012). For this game on the cycle the evolutionary dynamics can get trapped at intermediate numbers of mutants of either kind and fixation would also take infinitely long. There are coexistence game payoff configurations that permit mutant invasion, on the cycle, again when $b$ is the dominating payoff. In such cases, fixation times of the death-birth process tend to be much longer and experience a broader distribution than fixation times of the birth-death process.

Our analytical results focus on mean fixation times, but the simulation results highlight that future work has to focus on other features of the fixation time distribution, such as higher moments. Overall our work highlights that selective advantage might not guarantee fixation within a desired time frame, even on graphs that are deemed not to be a suppressor or amplifier of selection.

Acknowledgements

We thank Christian Hilbe, Kamran Kaveh and two anonymous referees for helpful comments.

P.M.A. gratefully acknowledges financial support from Deutsche Akademie der Naturforscher Leopoldina, grant no. LPDS 2012-12. A.T. is grateful for generous funding by the Max-Planck Society. M.A.N. was supported by the Bill and Melinda Gates Foundation [OPP1148627], the Office of Naval Research grant N00014-16-1-2914, National Cancer Institute grant CA179991 and by the John Templeton Foundation. The Program for Evolutionary Dynamics is supported in part by a gift from B. Wu and Eric Larson.

Appendix A Transition probabilities of the birth-death process

The transition probabilities of the birth-death process on the cycle graph for any strength of selection, in full form, are given by

[TABLE]

and

[TABLE]

Appendix B The probability to ever visit $j$, starting from $i$

The probabilities to ever go from any internal state $i$ to any other internal state $j$ are useful when calculating sojourn times Ewens (2004); Altrock et al. (2012),

[TABLE]

which hold for any birth-death process with absorbing boundaries (Goel and Richter-Dyn, 1974).

Appendix C Constant selection on cycle graphs

We speak of constant selection if the fitness of mutants is $f_{A}>1$ and the fitness of residents is $f_{B}\equiv 1$, irrespective of the number of mutants. Often, the success of the mutant is given by $r>1$, where the fitness of the resident is 1. We can then define $q=f_{B}/f_{A}=\text{e}^{-\beta r}$, and strong constant selection ($\beta\,r\to\infty$) in favor go the mutant means $q\to 0$. In the following we derive exact expressions for transition probabilities, sojourn times and the fixation time under strong constant selection.

In case of the birth-death process we find $q=T_{\text{Bd}}^{j-}/T_{\text{Bd}}^{j+}$ for all $j$ and get $\phi^{i+1,i}_{\text{Bd}}=1-(1-q)/(1-q^{N-i})$, and $\phi^{i,i+1}_{\text{Bd}}=(1-q^{i})/(1-q^{i+1})$. Then, we obtain the fixation probability Frean et al. (2013); Hindersin and Traulsen (2015); Askari and Samani (2015)

[TABLE]

as well as the sojourn times

[TABLE]

In the strong selection limit $q\to 0$ we obtain $T^{j+}_{\text{Bd}}\to 1/j$, and we can immediately see that

[TABLE]

and summing over all sojourn times case leads to

[TABLE]

which is equal to $t^{1\to N}_{\text{Bd}}$ as the mutant fixes with probability 1.

For the death-birth process, we obtain $T^{1-}_{\text{dB}}/T^{1+}_{\text{dB}}=(1+q)/2$, $T^{i-}_{\text{dB}}/T^{i+}_{\text{dB}}=q$ for $1<i<N-1$, and $T^{(N-1)-}_{\text{dB}}/T^{(N-1)+}_{\text{dB}}=2q/(1+q)$. Since the first and last state are special due to the structure of random death and subsequent competition, we obtain a more complicated expression for the fixation probability

[TABLE]

With the appropriate values of $\phi^{j+1,j}_{\text{dB}}$ and $\phi^{j,j+1}_{\text{dB}}$, the expected sojourn times become

[TABLE]

For strong selection, $q\to 0$, we observe that $\phi^{i}_{\text{dB}}\to 1$, $\phi^{j+1,j}_{\text{dB}}\to 0$ and $\phi^{j-1,j}_{\text{dB}}\to\text{const.}$ for all $j$, respectively. For the sojourn times we can now see that

[TABLE]

and hence the mean fixation time amount to

[TABLE]

Appendix D Constant selection approximation for the Bd process (cycle and well-mixed)

Here we are interested in finding approximations of the conditional mean fixation times for finite but large values of the product $\beta\,r$. In this case, we can perform a Taylor expansion to linear order in the quantity $q=\text{e}^{-\beta\,r}=f_{B}/f_{A}$, as defined above. For the well-mixed case we obtain $\phi^{1}_{\text{WM}}\approx 1-q$ and $\phi^{i>1}_{\text{WM}}\approx 1$, as well as $T_{\text{WM}}^{i-}/T_{\text{WM}}^{i+}=q$ for all $0<i<N$. To calculate the conditional mean fixation time of the well-mixed population we here resort to an equation that does not involve mean sojourn times Goel and Richter-Dyn (1974)

[TABLE]

and sort this equation in all terms involving directly $q^{0}$ or $q^{1}$

[TABLE]

whereby $s_{k}(q)=\phi_{\text{WM}}^{k}/T_{\text{WM}}^{k+}$. Now we use Eq. (12) with $\text{e}^{\beta\pi_{A}}=\text{e}^{\beta\,r}$ and $\text{e}^{\beta\pi_{B}}=1$, which is equivalent to setting the payoff $a=b=r/2$ and $c=d=0$, we get the linear in $q$ approximation

[TABLE]

to result in

[TABLE]

Then, the first sum of Eq. (113) amounts to

[TABLE]

and the second sum solves similarly. We thus obtain

[TABLE]

In case of the birth-death process on the cycle, we use the mean sojourn times from Eq. (100), the definition of the mean fixation time of Eq. (11), and $1/T_{\text{Bd}}^{k+}=k+(N-k)q$, to result in

[TABLE]