Power-law scaling of the effective population size in a branching particle system for moderate mutation-selection

TL;DR

This paper analyzes a branching Brownian motion model with a reflecting boundary, demonstrating that under certain conditions, the population's genealogy converges to Kingman's coalescent, contrasting with models leading to Bolthausen--Sznitman coalescent.

Contribution

It introduces a variant of branching Brownian motion with a reflecting boundary and proves the emergence of Kingman's coalescent in the large population limit.

Findings

Demonstrates polynomial time scale Yaglom law for population fluctuations.

Shows genealogy involves only binary mergers near the reflecting boundary.

Contrasts with models where genealogy follows Bolthausen--Sznitman coalescent.

Abstract

We consider a one-dimensional dyadic branching Brownian motion on $\mathbb{R}$ with positive drift $\beta \in (0,1)$, branching rate $1/2$, reflected at $0$ and killed at a boundary $L > 0$. The killing boundary $L$ is chosen so that the total population size remains approximately constant, proportional to $N \in \mathbb{N}$. This branching process models a population accumulating deleterious mutations. In the large-$N$ limit, we prove that when the typical width of the particle cloud is of order $c \log(N)$, with $c \in (0,1)$, the demographic fluctuations follow a Yaglom law on a polynomial time scale. Moreover, the limiting genealogy of the system involves only binary mergers, concentrated near the reflecting boundary. Our model is a version of the branching Brownian motion with absorption introduced by Berestycki, Berestycki, and Schweinsberg to study the effect of beneficial mutations on genealogies. In sharp contrast with their model, whose genealogy is given by a Bolthausen--Sznitman coalescent, we show that our system falls into the universality class of Kingman's coalescent.