Evolution under Stochastic Transmission: Mutation-Rate Modifiers

TL;DR

This paper investigates how stochasticity in genetic transmission, especially fluctuating mutation rates, influences evolutionary dynamics and the invasion success of mutation-rate modifiers, extending classical deterministic models.

Contribution

It extends the Reduction Principle to stochastic mutation rates, showing that random fluctuations can change the direction of selection on modifier alleles.

Findings

Stochastic mutation rates affect invasion success of modifiers.

Recombination influences the impact of mutation rate fluctuations.

Stochasticity can reverse deterministic predictions of mutation-rate evolution.

Abstract

Evolutionary analyses of large populations commonly incorporate stochasticity through temporal variation in selection while treating genetic transmission as fixed. Much less attention has been given to stochasticity in transmission itself. We study a selected locus with alleles $A$ and $a$ under constant selection, linked to a neutral modifier locus whose alleles $M_1$ and $M_2$ control the mutation rate from $A$ to $a$. Under constant transmission, the Reduction Principle applies: near a mutation--selection balance where $M_1$ is fixed with mutation rate $u_1$, a rare allele $M_2$ invades if its associated rate $u_2$ is smaller than $u_1$, but cannot invade if $u_2$ is larger than $u_1$. This result holds for both haploid and diploid populations and is independent of recombination, which affects only the rate, not the direction, of evolutionary change. We extend this framework by allowing the mutation rate associated with the invading modifier to fluctuate randomly across generations. In this stochastic setting, invasion is no longer determined by mean mutation rates alone. Instead, it depends on the temporal distribution of mutation rates, the strength of selection at the selected locus, and the recombination rate between modifier and target. Stochastic transmission and recombination therefore do not merely rescale deterministic predictions based on the Reduction Principle; they can alter the direction of selection on modifier alleles.