Game changing mutation

TL;DR

This paper models mutation in sexually reproducing populations as a game-theoretic process, showing how mutations can change evolutionary outcomes by altering the game's structure.

Contribution

The paper introduces a novel game-theoretic framework where mutations are modeled as changes to the game's action set, affecting evolutionary dynamics.

Findings

Populations converge to pure Nash equilibria in the absence of mutations.

Game-changing mutations can shift populations to new equilibria with higher fitness.

The model is applied to understand fitness valley crossing and evolutionary contingency.

Abstract

We present a model of the effect of mutation on haploid sexually reproducing populations by modelling the reproductive dynamics as occurring in the context of a common interests game played by the loci, with the alleles in the role of pure actions. Absent mutations, the population will deterministically converge to a pure Nash equilibrium of the game. A novel mutation adds new alleles, hence is tantamount to a change of the game by the addition of new actions. If the new game defined by the mutation removes the former pure Nash equilibrium the game changing mutation becomes in addition a Nash equilibrium changing mutation, as the population will then move to a new equilibrium with an increase in fitness. A graph of common interests games is defined, and evolution by mutation is modelled as a path through this graph. We discuss two applications—fitness valley crossing and evolutionary contingency.

Figures

A schematic representation of an allelic graph.

A schematic representation of a corresponding graph of population games.

Parallels between genotypic concepts and game theoretic concepts in this paper.

Illustration of added dimension removing equilibrium.

A plot of the game matrix 1 from example 1, demonstrating a fitness valley.

A plot of a section of game 2 in which the probabilities are equal across the diagonal, namely x(b1)=x(b2). Here, the valley of figure 5 is replaced by a steep climb.

Tables

Table 1.. Matrix 1. The pure Nash equilibria payoffs are highlighted in bold.

	b1	b2
a1	1	δ
a2	δ	2

Table 2.. Matrix 2. The pure Nash equilibria payoffs are highlighted in bold.

	b1	b2	b3
a1	1	δ	—
a2	δ	2	7/4

Table 3.. Matrix 3.

	b1	b2
a1	1	δ
a2	δ	2
a3	3	δ

Table 4.. Matrix 4.

	b1	b2	b3
a1	1	δ	3/2
a2	δ	2	3/2
a3	3	δ	δ

Equations

Full text

Introduction

The use of an evolutionary landscape, originally introduced in Wright [1,2], has become part of the conceptual foundation of evolutionary biology. The landscape especially provides an appealing scaffolding for modelling the role of evolution in adaptive dynamics and hence is appears in many papers on the topic. In a typical presentation of the main concept, one models a population with a dominant genotypic type A, alongside variants of lower fitness that are small in population weight relative to the dominant type. Subsequently, a mutation of an allele in what had been a lower fitness variant B grants that type higher fitness value than A. This shifts the population to be mostly composed of genotypes of type B, in the process lifting the population higher up the landscape.

The literature on fitness landscape exploration by mutations is vast (see [3,4]). It works particularly well for studying adaptation by mutation in models of asexual reproduction, where it is conceptually possible for a single allelic mutation to grant even a single individual in the population a fitness advantage that is directly transferred to descendants, enabling them to reproduce swiftly and take over the population.

By contrast, researchers have written about the difficulty of fitting models of sexual recombination to the adaptive landscape setting. They note that although on the one hand sexual recombination makes it likelier that beneficial mutations will become co-located in genotypes, on the other hand it can also undo beneficial combinations ‘because recently generated superior combinations are hard to maintain. They are likely to be lost because of recombination with other types’ ([5,6]). The result of the sexual dynamics is thus typically presented in the literature as ambiguous, depending on the question of which force is stronger: the one pushing beneficial alleles together or the one tearing apart good combinations?

To contend with the difficulties of fitting sexual reproduction models to the landscape models, many models in the literature have added assumptions over and above the landscape model itself, such as assuming small population bottlenecks ([7,8]), or specific heterogeneous distributions in the population ([9,10]).

We adopt a different and novel approach here to the study of mutation in sexually reproducing populations, moving away from the landscape, and modelling evolution as a path through an appropriately defined graph. We show that rather than an ambiguous tug-of-war between opposing forces, the sexual reproduction dynamics under full recombination leads asymptotically, in a deterministic trajectory depending on initial conditions, to a monomorphic population that is one of the Nash equilibria of an associated game. Mutations change the underlying game and possibly the equilibrium points as well, thus driving forward evolution.

Begin by noting that the main focus in the context of sexually reproducing populations is not on existing genotypes but rather on the collection $[eqn]$ of alleles available at each locus $[eqn]$ at time $[eqn]$ , because the potential genotypes that may exist in the population will be composed of those alleles in various combinations. We call the profile $[eqn]$ of available alleles, one per locus, an allelic formation. Regarding alleles as actions, we model sexual reproduction as a common interests game played by the loci: under pure strategies, each locus chooses an action/allele $[eqn]$ from $[eqn]$ , thus together forming a genotype $[eqn]$ . The payoff to forming genotype $[eqn]$ is the fitness value of that genotype. Extending this to mixed strategies, with the probability of allele $[eqn]$ being chosen equal to the prevalence of that allele in the population, yields the distribution of genotypes in the population at time $[eqn]$ .

Against this background we can give the sexual reproduction dynamics a geometric interpretation. The space of possible genotypes, conditional on an allelic formation $[eqn]$ , decomposes as a collection of basins of attraction in a space whose dimension is determined by the number of loci and the number of alleles per locus. The number of basins of attraction depends on the associated game $[eqn]$ , with one basin per pure Nash equilibrium of that game. Denoting by $[eqn]$ the set of pure Nash equilibria, under the sexual reproduction dynamics (without mutation), the population will move asymptotically towards one element of $[eqn]$ , conditional on the initial starting point, in a trajectory of increasing fitness payoff until a pure Nash equilibrium is attained. In summary, sexual reproduction asymptotically yields a monomorphic population bearing the genotype of one of the Nash equilibria of a common interests game.

This leads to the following insight: a novel mutation added to the allele set of locus $[eqn]$ at time $[eqn]$ is equivalent to the addition of a new action into the action set available to locus $[eqn]$ as a player in a game. After such a mutation the loci are actually playing another game, which expands the game previously played by way of the addition of actions (thus also moving the dynamics into a space of higher dimension). We call such a mutation a game changing mutation.

Crucially, not every game changing mutation will have a long-term effect on the population; that depends on how the set of pure Nash equilibria changes under the expansion of the game. Denote by $[eqn]$ the game played prior to the novel mutation and by $[eqn]$ the game following the mutation; furthermore denote by $[eqn]$ the Nash equilibrium genotype near which the pre-mutation population was located. If the geometry is such that $[eqn]$ is also a Nash equilibrium point in $[eqn]$ , then post mutation the population will remain within a basin of attraction of $[eqn]$ (in the space of higher dimension), and asymptotically return towards $[eqn]$ , leading to no significant effect of the mutation.

By contrast, if $[eqn]$ is not a Nash equilibrium of $[eqn]$ , a significant change will be apparent in the population as it will move away from $[eqn]$ to a point $[eqn]$ in $[eqn]$ . We term this event a Nash changing mutation. Under a Nash changing mutation, the fitness of the population increases as it moves towards a new Nash equilibrium point.

The model of mutations in sexually reproducing populations that emerges is of novel mutations expanding the sets of alleles available to populations. These mutations are all game changing in the sense that they change the underlying common interest game played by the loci, but not all such mutations move the population to a new equilibrium. Those that do so, the Nash changing mutations, will increase the population fitness in the process. In this picture, it is the Nash changing mutations that drive populations to new equilibria and higher fitness over time.

If we arrange the allelic formations in a directed graph, with an edge between two formations if and only if one is an expansion of the other by the addition of a single allele, evolution by mutations involves picking a path through this graph, via game changing or Nash changing mutations. Mutations that are not Nash changing have no long-term effect on the mean population fitness, but the Nash changing mutations only increase the mean fitness. With probability one, the path evolution will take through the graph will be one of increasing mean fitness over time.

In addition, we show that this model can shed light on questions relating to fitness valley crossing, and evolutionary contingency, relating to the question of whether the order of mutations affects evolutionary outcomes.

In summary outline form, the main points of the model here are as follows:

(1) Under the sexual reproduction dynamics, absent mutations, formations of alleles may be considered to define a common interests game, where the players are the loci and their available actions are the alleles.(2) The game divides the space of genotypes into disjoint basins of attraction. Population dynamics follow trajectories of monotonically increasing population mean fitness within these basins until the population converges to a Nash equilibrium of the game, which is a fixed point.(3) Sparse non-novel mutations have negligible effects on the sexual reproductive trajectories, because they do not move the population from one basin to another.(4) Novel mutations are game changers: by adding new alleles they change the actions available to the players, hence changing the game.(5) Not all game changing mutations, however, lead to population change. If the previous population fixed point, which was a Nash equilibrium of the previous game, is also a Nash equilibrium of the new game, the population will not move away from its previous equilibrium. One the other hand, if the previous equilibrium is not a Nash equilibrium of the new game, the population will move to a new equilibrium, in which case we term this event a Nash changing mutation.(6) With probability one, the path evolution will take is one of increasing mean fitness over time, from one game changing mutation to another at random arrival times.

Results

Overview of model of mutations

2.1.

From an abstract perspective, loci, which represent sites at which alleles are located, are the most elementary aspect of the model. Each locus $[eqn]$ is associated with a (possibly infinite) set $[eqn]$ of alleles. A genotype is a string of alleles, one at each locus. At any given time $[eqn]$ , however, there is only a finite set $[eqn]$ of alleles that are available at that time. The $[eqn]$ -tuple $[eqn]$ is termed the allelic formation at time $[eqn]$ .

A population at time $[eqn]$ is an ideal infinite collection of haploid genotypes, where each genotype $[eqn]$ satisfies the condition that for each $[eqn]$ , allele $[eqn]$ ; in other words, $[eqn]$ . The collection of all genotypes present at time $[eqn]$ when the allelic formation is $[eqn]$ is denoted by $[eqn]$ . Each genotype $[eqn]$ is associated with a fitness value $[eqn]$ , which is determined by $[eqn]$ but in this model is independent of the population state and the time1. For simplicity we will assume that in each possible $[eqn]$ there exists a unique genotype $[eqn]$ of maximal fitness.

Put together, we have all the ingredients for defining a common interests population game $[eqn]$ at time $[eqn]$ from $[eqn]$ , as follows. In this game, each locus $[eqn]$ is a player. The actions available to player/locus $[eqn]$ are the elements of the allele set $[eqn]$ . Each (pure) action profile $[eqn]$ precisely corresponds to a genotype as defined above, and the payoff to player $[eqn]$ when the action profile/genotype $[eqn]$ is played is $[eqn]$ . This forms an elementary aspect of our model: reproduction modelled as a common interests game between loci.

The allelic formations can be arranged into a directed graph, with an edge from $[eqn]$ to $[eqn]$ existing if and only if there is an $[eqn]$ such that $[eqn]$ for every $[eqn]$ , but $[eqn]$ is a singleton (figure 1). Since we have identified each allelic formation $[eqn]$ with a game $[eqn]$ , this graph can be perfectly mirrored in a corresponding graph of population games (figure 2).

A schematic representation of an allelic graph.

A schematic representation of a corresponding graph of population games.

Define the dimension of allelic formation $[eqn]$ to be the dimension of the simplex $[eqn]$ , where $[eqn]$ . Define the dimension of the corresponding game $[eqn]$ to be the same as the dimension of $[eqn]$ . Then by construction the dimension increases as one moves up the graph of games (figure 2).

A novel mutation is a single step in the graph of allelic formations in the direction of the arrows. A random walk through the graph yields a time-parametrized novel mutation process $[eqn]$ , associating an allelic formation with each point in time $[eqn]$ , from which one derives a corresponding time-parametrized process $[eqn]$ through the corresponding graph of games. The correspondence between the genotypic and game theoretic concepts is captured in figure 3.

Parallels between genotypic concepts and game theoretic concepts in this paper.

We note here a fact that will be important in the following: as one moves along a path up the population game graph, by construction all the actions available in games lower down the path are available in games farther up the path. However, this statement is not true for Nash equilibria: it is possible for a pair of games to be connected by an edge in the graph pointing from $[eqn]$ to $[eqn]$ such that a profile that is a Nash equilibrium of $[eqn]$ will not be a Nash equilibrium of $[eqn]$ , as the game geometry changes as dimensions are added. In other words, the set of Nash equilibria of $[eqn]$ may not necessarily contain the set of Nash equilibria of $[eqn]$ .

Figure 4 provides a simple illustration of the general idea that adding dimensions may erase previous equilibria. On the left, a point marked $[eqn]$ in a one dimensional simplex represents a stable equilibrium point, as forces on either side push any perturbation away from $[eqn]$ back to the equilibrium. On the right, the addition of a dimension embeds this one dimensional simplex in a two dimensional simplex. The additional dimension enables trajectories that move away from $[eqn]$ ; hence in the two dimensional simplex $[eqn]$ is not an equilibrium point.

Illustration of added dimension removing equilibrium.

Dynamics

2.2.

A game dynamic is a map that assigns to each game $[eqn]$ a differential equation $[eqn]$ , where $[eqn]$ is a population state and $[eqn]$ is the time derivative $[eqn]$ (see Sandholm [11]). If the game $[eqn]$ is not fixed but is instead given, for example, by a novel mutation process $[eqn]$ , one obtains from this the dynamics $[eqn]$ .

If $[eqn]$ and $[eqn]$ are the random arrival times of consecutive novel mutations, then the dynamic is determined by the same differential equation $[eqn]$ for all $[eqn]$ . In this case, for $[eqn]$ , the population state $[eqn]$ will take values in the simplex $[eqn]$ , whose dimension is fixed.

A random change then occurs at time $[eqn]$ : the law of motion will become $[eqn]$ and hence the population state $[eqn]$ will now take values in the higher dimensional simplex $[eqn]$ . The emerging dynamics is therefore a piecewise deterministic markov process (PDMP) [12,13]. However, whereas most PDMPs considered in applications take values in a space of fixed and bounded dimension (see Cloez et al. [14]) this PDMP takes values in spaces of varying and perhaps unbounded dimension. The model does not preclude non-novel mutations; a non-novel mutation at any time $[eqn]$ between two consecutive novel mutations in times $[eqn]$ can be captured by the deterministic dynamics of $[eqn]$ . For simplicity and coherence, we do not discuss such dynamics directly. However, some of our results show that the long run is mainly affected by novel mutations (see theorem 2.4).

Asexual and sexual reproduction dynamics

2.3.

We are interested in two types of states: genotypic and allelic. Genotypic states are probability distributions $[eqn]$ . A genotypic state is monomorphic if there is a genotype $[eqn]$ such that $[eqn]$ . An *allelic state of locus $[eqn]$ * is a probability distribution $[eqn]$ . An allelic state is $[eqn]$ . A genotypic state has an associated allelic state: denote by $[eqn]$ the proportion of genotypes $[eqn]$ carrying allele $[eqn]$ in locus $[eqn]$ , i.e.

$$[eqn]$$

In models of asexual reproduction, genotypic states are typically the main focus of attention. The dynamics in such models is the replicator dynamics [15]:

$$[eqn]$$

where $[eqn]$ is the population mean fitness and the growth rate of $[eqn]$ is $[eqn]$ . As $[eqn]$ , the mean fitness is a potential function of individual fitness and the population game in this case is a potential game (see Sandholm [11]).

In models of sexual reproduction, allelic states are typically the main focus of attention. We assume that there is no correlation across alleles. In allelic state $[eqn]$ the population mean fitness is given by $[eqn]$ , where $[eqn]$ means that allele $[eqn]$ is in locus $[eqn]$ of $[eqn]$ . The individual payoff of player/locus $[eqn]$ if she plays $[eqn]$ (namely, allele $[eqn]$ in locus $[eqn]$ ) is

$$[eqn]$$

Similar to the above, $[eqn]$ ; hence the mean fitness function $[eqn]$ is a potential function for the game whose players are the loci.

If an allelic-formation $[eqn]$ is kept fixed, then since the dynamics is defined via a potential function, it is natural to ask whether the dynamics is executing a gradient climb. For genotypic states equation (2.1) it was shown by Shahshahani [16] that when taking into consideration the appropriate metric2, this is indeed the case.

Theorem 2.1 (Shahshahani [16]). The replicator dynamics of a potential game is a Shahshahani gradient in the interior of the simplex.

From Shahshahani’s theorem it follows that in models of asexual reproduction without mutation, from any internal point in the simplex the population follows a trajectory of monotonically increasing mean fitness along the replicator dynamics, asymptotically approaching a monomorphic population comprising the genotype of maximal fitness.

A similar result holds for sexually reproducing populations, as we show here. In the sexual reproduction model, two individuals mate to produce offspring. When an individual with genotype $[eqn]$ mates with an individual with genotype $[eqn]$ , the genotype of an offspring $[eqn]$ satisfies the following property: for each $[eqn]$ , allele $[eqn]$ equals either $[eqn]$ or $[eqn]$ with equal probability. The probability that an individual with genotype $[eqn]$ will successfully mate and produce offspring is proportional to $[eqn]$ ; in other words, fitness is defined to be the probability of reproduction.

We denote by $[eqn]$ the proportion of genotypes bearing the $[eqn]$ th allele in the $[eqn]$ th locus, and by $[eqn]$ the marginal fitness of that allele when the allelic state is $[eqn]$ . Note that we are tracking here the proportions of alleles at each locus separately. Instead of one simplex to follow, there are $[eqn]$ simplices, one simplex $[eqn]$ for each locus $[eqn]$ . Define the multi-simplex $[eqn]$ ; this is the state space of the sexual reproduction model.

Within each simplex $[eqn]$ the alleles internal to locus $[eqn]$ are competing with each other. However, the marginal fitness of each allele in locus $[eqn]$ at each point in time is also a function of the full state of the population in the multi-simplex. The resulting multi-replicator dynamics is given3 by

$$[eqn]$$

Despite the complexity of the sexual reproduction dynamic, even when the allelic formation $[eqn]$ is kept fixed, it turns out that under an appropriate metric4 sexual reproduction executes a straightforward gradient climb

Theorem 2.2. For a fixed allelic formation $[eqn]$ , the multi-replicator dynamics is a multi-Shahshahani gradient over a potential population game in the interior of the multi-simplex $[eqn]$ .

Theorem 2.3. From any point in the interior of the multi-simplex, a sexually reproducing population with allelic formation $[eqn]$ converges asymptotically to a monomorphic population bearing a genotype that is in $[eqn]$ , the set of pure Nash equilibria of the associated common interests game.

The exact definitions used in these theorems, and the proofs, can be found in appendix A [17]. Intuitively, the common interests game divides the multi-simplex into disjoint basins of attraction, one basin of attraction for each pure Nash equilibrium in $[eqn]$ . The population follows the multi-replicator dynamics, which determines gradient-climbing trajectories of monotonically increasing population mean fitness within each basin until the population converges to a Nash equilibrium of the game, which is a fixed point5.

There are immediate significant implications to these theorems: in asexually reproducing populations the mean fitness not only increases monotonically, but that it does so at a maximal possible rate of ascent. This mean fitness increase continues until the global maximum of the (in this case linear) mean fitness function over the convex simplex is attained, generically at a corner solution. Under this dynamics, for any $[eqn]$ the population will generically be $[eqn]$ -clustered around the genotype $[eqn]$ of maximal fitness in finite time, and asymptotically will converge to $[eqn]$ .

In sexually reproducing populations, the mean fitness also increases monotonically at a maximal rate of ascent, but now the mean fitness increase continues until the population asymptotically arrives at the local maximum of a basin attraction. Similar to the above, in finite time, for any $[eqn]$ the population will generically be $[eqn]$ -clustered around the genotype $[eqn]$ of maximal fitness within one of the basins of attraction.

Chastain et al. [21] also studied the relationship between games and sexual reproduction. Their model studies a discrete time algorithm, the multiplicative weights updating algorithm (MWUA), and shows that it corresponds to sexual reproduction under weak selection. The model there indicates that if the dynamics converges then sexual reproduction will achieve maximal fitness. However, the authors there do not show that the dynamics actually converges, nor that it constantly increases fitness. Although the MWUA is often called ‘the discrete replicator’ it is important to note that in the no-regret regime studied in Chastain et al. [21] the MWUA is not a discrete approximation of the replicator dynamics.

Game changing and Nash changing mutations

2.4.

In the model of this paper, relatively rare non-novel mutations have no significant effects on near equilibrium states of sexually reproducing populations. This is because such mutations can slightly change the distributions of the genotypes in the population, but if sufficiently rare they will not move the population out of one basin of attraction to another basin. As long as the population remains in the same basin of attraction, all the trajectories converge to the same pure Nash equilibrium of the associated game. We therefore concentrate from here on novel mutations.

In a slightly simplified version of the novel mutation process, we presume the existence of a Poisson clock with parameter $[eqn]$ . When the clock goes off at a random time $[eqn]$ , a locus $[eqn]$ is selected at random together with $[eqn]$ , generating in this way a random walk through the allelic formations graph6.

Consider two separate times $[eqn]$ and $[eqn]$ . We denote the allelic formation at $[eqn]$ (resp. $[eqn]$ ) by $[eqn]$ (resp. $[eqn]$ ), with associated game $[eqn]$ (resp. $[eqn]$ ). We also concentrate on genotypes $[eqn]$ and $[eqn]$ , respectively, where $[eqn]$ is a Nash equilibrium of the game $[eqn]$ while $[eqn]$ is a Nash equilibrium of the game $[eqn]$ .

At time $[eqn]$ , the allelic formation in the population is $[eqn]$ and the population is very nearly monomorphic at $[eqn]$ . Prior to $[eqn]$ no mutation occurs, and the allelic formation remains fixed, i.e. $[eqn]$ . The population state remains near $[eqn]$ .

At random arrival time $[eqn]$ a mutation occurs, locus $[eqn]$ is selected, and allele $[eqn]$ is added to $[eqn]$ , yielding $[eqn]$ . If $[eqn]$ , no change has occurred: $[eqn]$ , and the same population game as before continues to be played. In this case, the population will not move away from its previous equilibrium point: $[eqn]$ .

Suppose instead that $[eqn]$ . Then $[eqn]$ , implying that a new game $[eqn]$ is being played. We term this a game changing mutation event.

This, however, does not necessarily mean that the population point moves to a new equilibrium point. The key question is whether $[eqn]$ , which is a Nash equilibrium of $[eqn]$ is also a Nash equilibrium of $[eqn]$ . If yes, then the population will remain near $[eqn]$ even after the game changing mutation. If no, since the population state which was a Nash equilibrium prior to the mutation is not a Nash equilibrium of the new game, the population will move towards the new equilibrium point at $[eqn]$ . In that case, we say that a Nash changing mutation event has occurred.

This has implications for the population fitness level. If $[eqn]$ , the fitness level will remain at $[eqn]$ . If $[eqn]$ then $[eqn]$ as the population moves to a new Nash equilibrium.

In summary, the following picture emerges. Evolution is modelled as a random connected path $[eqn]$ through the allelic graph7. At each point $[eqn]$ in a series of random times $[eqn]$ , the population moves from $[eqn]$ to $[eqn]$ .

If $[eqn]$ , any mutation that has occurred is not a novel mutation. In this case, $[eqn]$ and $[eqn]$ . If $[eqn]$ , a game changing mutation has occurred. In this case, there are two possibilities: either $[eqn]$ and $[eqn]$ , or $[eqn]$ and $[eqn]$ . The latter event is a Nash changing mutation. As the population moves through the path $[eqn]$ , the mean population fitness generically increases monotonically.

This intuition is formalized in the following result (proved rigorously in the appendix A). This result has further implications for convergence to a monomorphic population, which is associated with pure Nash equilibrium of the underlying common interest game.

Theorem 2.4. Following a Nash changing mutation, the mean population fitness generically increases monotonically.

Theorem 2.5. A population reproducing under the haploid sexual reproduction dynamics will with probability 1 follow a path through the allelic graph of (possibly weakly) monotonically increasing mean population fitness payoff.

Theorem 2.5 summarizes the main conclusion of our model: in haploid sexually reproducing populations, accumulating novel mutations that monotonically expand allelic formations lead to increasing fitness values.

Recall that the set of pure Nash equilibria of a game $[eqn]$ is denoted $[eqn]$ . If novel mutations are rare, namely $[eqn]$ is sufficiently small, and if $[eqn]$ are two consecutive arrival times of novel mutations, one expects that with high probability in time $[eqn]$ that is sufficiently near time $[eqn]$ , the state $[eqn]$ will be sufficiently near a Nash equilibrium of $[eqn]$ . To capture this, denote by $[eqn]$ the distance of a point $[eqn]$ from the set of Nash equilibria of the game $[eqn]$ . We then have the following result.

Theorem 2.6. Let $[eqn]$ be two consecutive arrival times of novel mutations. For every $[eqn]$ there are $[eqn]$ and a random variable $[eqn]$ such that $[eqn]$ almost-surely and for almost-every realization $[eqn]$ we have that for every $[eqn]$ and every $[eqn]$

$$[eqn]$$

Fitness valley crossing

2.5.

Raising fitness values may require combinations of mutations at different loci, since it is the interactions between alleles in the loci that determine fitness (examples from the literature include complex signalling pathways, and multiple mutations that may be needed to metabolize nutrients). It is possible that successive individually beneficial mutations can effect a monomorphic fitness climb [22], but it is likelier that successful adaptations require combinations of mutations which individually are deleterious. When this is the case, it is said that populations need to cross a ‘fitness valley’ [1].

In this section, we build on the haploid sexual reproduction model from the previous sections. Our results show that fitness valley crossing in our model may only require a single mutation, whereas common wisdom is that it requires at least two (e.g. [23,24]).

A fitness valley is given by two local maxima of the landscape $[eqn]$ , say $[eqn]$ and $[eqn]$ such that

(a) $[eqn]$ ,(b) the Hamming distance $[eqn]$ between the genotypes $[eqn]$ and $[eqn]$ is at least two, namely, at least two mutations are required to move from $[eqn]$ to $[eqn]$ ,(c) there is no local maximum $[eqn]$ such that8 $[eqn]$ .

In the standard fitness landscape model of evolutionary biology, the allele formation $[eqn]$ is kept fixed. Thus if $[eqn]$ is a local maximum and the population state lies sufficiently near it then the population will converge to $[eqn]$ . In the event of mutation that changes a single allele of $[eqn]$ , the population state will be slightly perturbed initially, but will nevertheless continue to converge to $[eqn]$ ; this is a corollary of theorem 2.2. Thus a single mutation event that slightly changes the weights within the genotype space $[eqn]$ will not lead to a convergence to the improved maximum $[eqn]$ . This is referred to as the problem of crossing the fitness valley, the metaphor being of a landscape in a local fitness hill surrounded by a valley of lower fitness. As we have illustrated, in this model a population near a local fitness maximum will never climb to higher fitness values short of an extremely rare occurrence of two mutations within one individual.

Contrary to this, game changing mutations in a sexually reproducing population can dramatically and rapidly change the composition of a population. This can give sexually reproducing populations advantages over asexually reproducing populations.

Example 1. Let $[eqn]$ and $[eqn]$ and let $[eqn]$ and $[eqn]$ .

Suppose that under initial conditions, $[eqn]$ and $[eqn]$ . The fitness landscape, with fitness function $[eqn]$ , is illustrated in table 1 (with the Nash equilibria shown in bold). It is captured graphically in figure 5. This defines an allelic formation $[eqn]$ .

A plot of the game matrix 1 from example 1, demonstrating a fitness valley.

Suppose that the initial population state $[eqn]$ places weight $[eqn]$ on genotype $[eqn]$ , and weight $[eqn]$ on each of $[eqn]$ and $[eqn]$ , with weight $[eqn]$ on $[eqn]$ (where $[eqn]$ is extremely small). In an asexually reproducing population, genotype $[eqn]$ will maintain its central position in the population, leaving only trace amounts of individuals bearing genotypes $[eqn]$ and $[eqn]$ . The mean population fitness value will be very close to one.

A sexually reproducing population will look very similar, apart from the fact that by random mating there will be small non-zero weight on genotype $[eqn]$ . A basin of attraction around genotype $[eqn]$ will exist and the mean population fitness value will be very close to one.

Next, suppose that a mutation event occurs, with an individual bearing genotype $[eqn]$ mutating to $[eqn]$ . The fitness landscape then changes to that illustrated in table 2.

For an asexually reproducing population, this will have the effect of changing the state of the population to being nearly monomorphically $[eqn]$ , and the mean population fitness value will be nearly $[eqn]$ .

The result in the sexually reproducing population will be quite different. The allelic formation expands to $[eqn]$ . As table 2 indicates, $[eqn]$ is not a Nash equilibrium of the expanded game. Since the dynamics must carry the population towards a Nash equilibrium, the population will move away from $[eqn]$ and will eventually be nearly monomorphically composed of genotype $[eqn]$ . The mean population fitness value will be nearly $[eqn]$ .

It is interesting to note that in this example although the mutation to allele $[eqn]$ kicked off the process, its appearance eventually boosts allele $[eqn]$ asymptotically to fixation while $[eqn]$ itself asymptotically goes extinct, as illustrated in figure 6. An observer who views the population when it is mainly composed of genotype $[eqn]$ and later views it when it is mostly $[eqn]$ may not have an indication that a mutation to $[eqn]$ was involved at all.

A plot of a section of game 2 in which the probabilities are equal across the diagonal, namely x(b1)=x(b2). Here, the valley of figure 5 is replaced by a steep climb.

The previous example exhibited a situation in which sexual reproduction gains an advantage over asexual reproduction by way of game changing mutations. The sexual reproduction dynamics, however, can also be disadvantageous in certain situations. Its main weakness is that a mutation that does not remove a Nash equilibrium around which a population is clustered will not move the population to a new state, even if that mutation is highly beneficial for the genotype in which the mutation has occurred. The next example exhibits this.

Example 2. This example is very similar in its initial conditions to example 1. Let $[eqn]$ and $[eqn]$ , $[eqn]$ . The fitness landscape is again that of table 1.

Suppose again that the initial population state $[eqn]$ places weight $[eqn]$ on genotype $[eqn]$ , and weight $[eqn]$ on each of $[eqn]$ and $[eqn]$ , with weight $[eqn]$ on $[eqn]$ (where $[eqn]$ is extremely small). As before, both the asexual and sexual populations are clustered around genotype $[eqn]$ with mean population fitness near one (in the sexually reproducing population, however, there will rapidly be small population weight developing on genotype $[eqn]$ , via recombination, as opposed to the zero weight on $[eqn]$ of the asexual population).

Suppose now that an individual in the population bearing genotype $[eqn]$ undergoes a mutation to $[eqn]$ . In the sexually reproducing population, nothing changes: there was already a small weight of individuals with genotype $[eqn]$ , hence the mean population fitness remains near one. In the asexual population, however, this mutation brings about a dramatic change: the introduction of an individual of genotype $[eqn]$ leads asymptotically to that genotype taking over the population, driving the mean population fitness towards 2.

Evolutionary contingency

2.6.

Both asexual and sexual reproduction strive to find maximal-fitness solutions. However, this process may not be deterministic as the outcomes may also depend on idiosyncratic events that an evolving lineage experiences such as the order of appearance of random mutations. Should the tape of life be replayed, would it produce similar living beings? This question known as historical contingency, or contingency for short, was argued by Stephen Jay Gould [25] to be an essential feature of evolution.

Gould’s original idea introduced confusion regarding the notion of contingency and the way it operates [26]. Some authors have tried to resolve this confusion (e.g. [26–30]), but these papers generally did not present thorough modelling frameworks.

In the standard models of asexual mutation in the literature, the order in which a chain of mutations occurs makes no difference to the end resulting genotype (although it could affect the chances that a particular chain will arrive at the endpoint). Consider, for example, two chains of mutations (i) one taking the population from monomorphic $[eqn]$ to $[eqn]$ and then to $[eqn]$ ; (ii) alternatively another path of genotypes moving the population from $[eqn]$ to $[eqn]$ and then to $[eqn]$ . In both cases, the end result is $[eqn]$ and the payoff is that of the genotype $[eqn]$ .

In the model of haploid sexual reproduction of this paper, however, the end result of a chain of mutations may be very dependent on the order in which mutations occur, as the population may climb the genotypic lattice through different paths from one point to another. This is exhibited in the following example.

Example 3. Recall the matrices in tables 1 and 2 above. We have already established that when table 1 is augmented to table 2 the sexually reproducing population will move, from a state close to $[eqn]$ , to $[eqn]$ .

Consider now the possibility of Matrix 1 being augmented instead to Matrix 3 (see table 3). In the move from Matrix 1 to Matrix 3, the state $[eqn]$ is no longer a Nash equilibrium. The population will instead move to the state $[eqn]$ .

Finally, consider Matrix 4 (see table 4). If Matrix 2 is augmented to Matrix 4, from a state close to the $[eqn]$ , the population will remain in the vicinity of $[eqn]$ . Similarly, if Matrix 3 is augmented to Matrix 4, from a state close to the $[eqn]$ , the population will remain in the vicinity of $[eqn]$ .

Hence we have shown the path dependence of the game changing mutation process: adding $[eqn]$ and then $[eqn]$ does not lead to the same result as adding $[eqn]$ first and then $[eqn]$ . The former represents moving from Matrix 1 to Matrix 2 to Matrix 4, hence from $[eqn]$ to $[eqn]$ , while the latter moving from Matrix 1 to Matrix 3 to Matrix 4, hence from $[eqn]$ to $[eqn]$ .

Robustness under random arrivals

2.7.

A powerful recasting of evolutionary contingency is the predictability of evolutionary outcomes [31]: if life’s tape is replayed can we make predictions about what to expect? In the next two sections we explore two matters related to predictability. The first, which we discuss here, is the robustness of the order of arrival of novel mutations to randomness.

Our previous examples concentrated on realized arrival orders of mutations, illustrating the underlying idea of game changing and Nash changing mutations. Here we extend the discussion to accommodate random arrivals of mutations and evaluate the probability of different contingencies.

We revisit example 3, by assuming that the arrivals of mutations for actions $[eqn]$ and $[eqn]$ follow exponential distributions with parameters $[eqn]$ and $[eqn]$ , respectively. Let $[eqn]$ and $[eqn]$ denote the random arrival time of actions $[eqn]$ and $[eqn]$ , respectively. As we have seen, the resulting Nash equilibrium for the event $[eqn]$ will be different from that of the event $[eqn]$ . To evaluate the probability of each Nash equilibrium, one needs to evaluate the probability of the aforementioned inequalities between arrival times:

$$[eqn]$$ $$[eqn]$$

and by symmetry $[eqn]$ .

It is easy to see that, for exampl as $[eqn]$ we have $[eqn]$ , namely, as the arrival time of $[eqn]$ becomes large compared with that of $[eqn]$ , the probability of the Nash equilibrium moving from $[eqn]$ to $[eqn]$ tends to $[eqn]$ , showing the robustness of example 3 in this case.

Replaying the tape of life

2.8.

What does assessing the ‘predictability’ of outcomes mean? Example 3 showed how differences in fitness emerge from different orders of mutations. Here we aim at finding a quantitative estimation of this effect. As we shall see, the measurement of fitness is not the only important thing—measurement timing is important as well.

Let $[eqn]$ be the random arrival times of novel mutations, and define

$$[eqn]$$

Hence $[eqn]$ defines the random walk of fitness due to novel mutations. As $[eqn]$ is a martingale, we could apply the martingale CLT to study it. Suppose that fitness jumps are bounded, namely there is a $[eqn]$ such that $[eqn]$ for every $[eqn]$ . Define $[eqn]$ and let

$$[eqn]$$

Then

Theorem 2.7. The random variable $[eqn]$ converges in distribution to the normal distribution with mean $[eqn]$ and variance $[eqn]$ as $[eqn]$ .

Notice that the measurement timing is important for the result. The measurement time $[eqn]$ itself is a random variable. Furthermore, the process contains randomness from two different sources:

(1) the random order of arrival of novel mutations; and(2) the distance of the state $[eqn]$ from the respective pure Nash equilibrium.

As $[eqn]$ one would expect the randomness due to item (2) to diminish, hence we would be left with the randomness due to random order of arrival of random mutations.

Conclusion and further questions

We have introduced a model of mutation as stochastic movement through the allelic lattice under the haploid sexual reproduction dynamic. This leaves us with several questions for further research:

(1) The model here presumes that novel alleles are always added to existing allelic formations; alleles never disappear. This is unrealistic. A model in which alleles can appear over time and also disappear is needed. In such a model, paths through the allelic graph will not be unidirectional, and at present it is unclear what long-term results with respect to mean population fitness values could be expected; further assumptions to the model may be necessary.(2) What qualitative long-term convergence results can be expected in the model of this paper? Benaïm et al. [13] developed such a theory for certain PDMPs with bounded dimension. Our work here introduces a new example of PDMP whose ‘dimension’ (measured here by the size of the allelic formation) may be unbounded as $[eqn]$ may be infinite. Developing such a theory for PDMPs similar to the one we have discussed is beyond the scope of this paper, and will be pursued in a subsequent work.(3) How likely are new Nash equilibria to appear as games are expanded? This is a question that is of interest in game theory in general, not only with respect to evolutionary theory: as games expand, as defined in this paper, should one expect new Nash equilibria to appear often or rarely? The answer may depend on specific structural aspects of the games involved. A characterization of such aspects would be a contribution to the literature.(4) How much of what is presented here survives in models with frequency dependent fitness, or in models of finite populations? A model with frequency dependence will likely not maintain the structure of a strategic game on which many of the results here depend—the less demanding concept of a population game will be relevant. In a finite population model, genetic drift effects may become prominent to the point that they significantly change the convergence results presented in the body of this paper.