Sample genealogy and mutational patterns for critical branching populations

TL;DR

This paper models the genealogy of critical branching populations with mutations using a Poissonian birth-death process, providing explicit mutational pattern formulas and convergence results for large samples under various priors.

Contribution

It introduces a universal genealogical model for critical branching populations with mutations, extending to random foundation times and deriving explicit mutational spectra and convergence properties.

Findings

Explicit formulas for the expected site frequency spectrum.

Convergence in distribution of large sample genealogies.

All limiting genealogies can be embedded in a single Poisson point measure.

Abstract

We study a universal object for the genealogy of a sample in populations with mutations: the critical birth-death process with Poissonian mutations, conditioned on its population size at a fixed time horizon. We show how this process arises as the law of the genealogy of a sample in a large class of critical branching populations with mutations at birth, namely populations converging, in a large population asymptotic, towards the continuum random tree. We extend this model to populations with random foundation times, with (potentially improper) prior distributions g_i: x\mapsto x^{-i}, i\in\Z_+, including the so-called uniform (i=0) and log-uniform (i=1) priors. We first investigate the mutational patterns arising from these models, by studying the site frequency spectrum of a sample with fixed size, i.e. the number of mutations carried by k individuals in the sample. Explicit formulae for the expected frequency spectrum of a sample are provided, in the cases of a fixed foundation time, and of a uniform and log-uniform prior on the foundation time. Second, we establish the convergence in distribution, for large sample sizes, of the (suitably renormalized) tree spanned by the sample genealogy with prior g_i on the time of origin. We finally prove that the limiting genealogies with different priors can all be embedded in the same realization of a given Poisson point measure.