Combinatorics of ancestral lines for a Wright-Fisher diffusion with selection in a L\'evy environment

TL;DR

This paper introduces a novel combinatorial method to analyze the fixation probabilities in a Wright-Fisher diffusion model with complex two-sided selection in a Lévy environment, overcoming limitations of classical approaches.

Contribution

It develops a new encoding of the ancestral selection graph's combinatorics into a function, leading to differential equations and series representations for fixation probabilities.

Findings

Derived a series expansion for fixation probability $h(x)$.

Established a linear system of differential equations for coefficients.

Provided explicit recursion formulas for series coefficients.

Abstract

Wright-Fisher diffusions describe the evolution of the type composition of an infinite haploid population with two types (say type $0$ and type $1$) subject to neutral reproductions, and possibly selection and mutations. In the present paper we study a Wright-Fisher diffusion in a L\'evy environment that gives a selective advantage to sometimes one type, sometimes the other. Classical methods using the Ancestral Selection Graph (ASG) fail in the study of this model because of the complexity, resulting from the two-sided selection, of the structure of the information contained in the ASG. We propose a new method that consists in encoding the relevant combinatorics of the ASG into a function. We show that the expectations of the coefficients of this function form a (non-stochastic) semigroup and deduce that they satisfy a linear system of differential equations. As a result we obtain a series representation for the fixation probability $h(x)$ (where $x$ is the initial proportion of individuals of type $0$ in the population) as an infinite sum of polynomials whose coefficients satisfy explicit linear relations. Our approach then allows to derive Taylor expansions at every order for $h(x)$ near $x=0$ and to obtain an explicit recursion formula for the coefficients.