Asymptotic behaviour of sampling and transition probabilities in coalescent models under selection and parent dependent mutations

TL;DR

This paper investigates the asymptotic properties of sampling and transition probabilities in coalescent models with selection and parent-dependent mutations, providing new insights into their behavior as sample size grows large.

Contribution

It derives asymptotic results for sampling and transition probabilities in coalescent models with parent-dependent mutations and selection, which were previously not explicitly known.

Findings

Sampling probabilities decay polynomially with sample size

Transition probabilities converge to type frequency limits

Results depend on stationary density of Wright-Fisher diffusion

Abstract

The results in this paper provide new information on asymptotic properties of classical models: the neutral Kingman coalescent under a general finite-alleles, parent-dependent mutation mechanism, and its generalisation, the ancestral selection graph. Several relevant quantities related to these fundamental models are not explicitly known when mutations are parent dependent. Examples include the probability that a sample taken from a population has a certain type configuration, and the transition probabilities of their block counting jump chains. In this paper, asymptotic results are derived for these quantities, as the sample size goes to infinity. It is shown that the sampling probabilities decay polynomially in the sample size with multiplying constant depending on the stationary density of the Wright-Fisher diffusion and that the transition probabilities converge to the limit of frequencies of types in the sample.