Branching Brownian motion and Selection in the Spatial Lambda-Fleming-Viot Process

TL;DR

This paper investigates how spatial structure affects the detectability of natural selection in populations, revealing that in two dimensions, stronger selection signals are needed for detection due to spatial constraints.

Contribution

It introduces a spatial ancestral selection graph model and analyzes its scaling limits, showing dimension-dependent effects on natural selection detectability.

Findings

In dimensions ≥3, spatial structure minimally impedes selection detection.

In 2D, stronger selection is required for detection, scaled by log(1/μ).

The ancestral process converges to a branching Brownian motion under certain conditions.

Abstract

We ask the question "when will natural selection on a gene in a spatially structured population cause a detectable trace in the patterns of genetic variation observed in the contemporary population?". We focus on the situation in which 'neighbourhood size', that is the effective local population density, is small. The genealogy relating individuals in a sample from the population is embedded in a spatial version of the ancestral selection graph and through applying a diffusive scaling to this object we show that whereas in dimensions at least three, selection is barely impeded by the spatial structure, in the most relevant dimension, $d=2$, selection must be stronger (by a factor of $\log(1/\mu)$ where $\mu$ is the neutral mutation rate) if we are to have a chance of detecting it. The case $d=1$ was handled in Etheridge et al. (2015). The mathematical interest is that although the system of branching and coalescing lineages that forms the ancestral selection graph converges to a branching Brownian motion, this reflects a delicate balance of a branching rate that grows to infinity and the instant annullation of almost all branches through coalescence caused by the strong local competition in the population.