A conditional coalescent for diploid exchangeable population models given the pedigree

TL;DR

This paper analyzes the genealogical structure of diploid populations conditioned on their pedigree, revealing significant differences from classical models and characterizing a new inhomogeneous coalescent process.

Contribution

It introduces an inhomogeneous $( ext{ extPsi},c)$-coalescent model that accounts for fixed pedigrees, extending prior work by incorporating pedigree effects into genealogical analysis.

Findings

Conditional coalescent processes differ from classical models with multiple mergers.

The limiting process is characterized as an inhomogeneous $( ext{ extPsi},c)$-coalescent.

Pedigree structure impacts multi-locus genetic statistics.

Abstract

We study coalescent processes conditional on the population pedigree under the exchangeable diploid bi-parental population model of \citet{BirknerEtAl2018}. While classical coalescent models average over all reproductive histories, thereby marginalizing the pedigree, our work analyzes the genealogical structure embedded within a fixed pedigree generated by the diploid Cannings model. In the large-population limit, we show that these conditional coalescent processes differ significantly from their marginal counterparts when the marginal coalescent process includes multiple mergers. We characterize the limiting process as an inhomogeneous $(\Psi,c)$-coalescent, where $\Psi$ encodes the timing and scale of multiple mergers caused by generations with large individual progeny (GLIPs), and $c$ is a constant rate governing binary mergers. Our results reveal fundamental distinctions between quenched (conditional) and annealed (classical) genealogical models, demonstrate how the fixed pedigree structure impacts multi-locus statistics such as the site-frequency spectrum, and have implications for interpreting patterns of genetic variation among unlinked loci in the genomes of sampled individuals. They significantly extend the results of \citet{DiamantidisEtAl2024}, which considered a sample of size two under a specific Wright-Fisher model with a highly reproductive couple, and those of \citet{TyukinThesis2015}, where Kingman coalescent was the limiting process. Our proofs adapt coupling techniques from the theory of random walks in random environments.