$\Lambda$-Wright--Fisher processes with general selection and opposing environmental effects: fixation and coexistence

TL;DR

This paper analyzes the long-term behavior of $ ext{Lambda}$-Wright--Fisher processes with complex selection and environmental effects, revealing new phenomena like boundary repulsion and coexistence, with explicit criteria and probabilistic representations.

Contribution

It introduces a novel analysis of boundary behaviors in $ ext{Lambda}$-Wright--Fisher processes using Siegmund duality, addressing models previously intractable by traditional methods.

Findings

Boundary points can be repelling, enabling coexistence.

Explicit criteria distinguish between different boundary behaviors.

A new probabilistic representation of fixation probability is provided.

Abstract

Our results characterize the long-term behavior for a broad class of $\Lambda$-Wright--Fisher processes with frequency-dependent and environmental selection. In particular, we reveal a rich variety of parameter-dependent behaviors and provide explicit criteria to discriminate between them. That includes the situation in which the (entire) boundary is repelling -- a new phenomenon in this context. This has significant biological implications, because it means that selection alone can maintain coexistence. If a boundary point is attractive, we derive polynomial/exponential decay rates for the probability of not being polynomially/exponentially close to that boundary, depending on some weak/strong integrability conditions. Moreover, we provide a handy representation of the fixation probability. In our proofs we make use of Siegmund duality. The dual process can be sandwiched near the boundaries in between transformed L\'evy processes. In this way we relate the boundary behavior of the dual process to fluctuation properties of these L\'evy processes and shed new light on previously established conditions for attractive/repelling boundary points. Our method allows us to treat models that so far could not be analyzed by means of moment or Bernstein duality. This closes an existing gap in the literature.