The Distribution of Fst and other Genetic Statistics for a Class of Population Structure Models

TL;DR

This paper introduces G/KC population structure models, deriving the distribution of Fst and other genetic statistics in large populations, unifying well-known models and extending analysis to homozygosity measures using coalescent methods.

Contribution

It defines a new class of population models, G/KC, and derives a general formula for Fst distribution applicable to multiple models, including island and stepping stone models.

Findings

Derived Fst distribution for G/KC models in large populations

Unified analysis of island and stepping stone models as G/KC members

Extended methodology to homozygosity and other genetic statistics

Abstract

We examine genetic statistics used in the study of structured populations. In a 1999 paper, Wakeley observed that the coalescent process associated with the finite island model can be decomposed into a scattering phase and a collecting phase. In this paper, we introduce a class of population structure models, which we refer to as G/KC models, that obey such a decomposition. In a large population, large sample limit we derive the distribution of the statistic Fst for all G/KC models under the assumptions of strong or weak mutation. We show that in the large population, large sample limit the island and two dimensional stepping stone models are members of the G/KC class of models, thereby deriving the distributions of Fst for these two well known models as a special case of a general formula. We show that our analysis of Fst can be extended to an entire class of genetic statistics, and we use our approach to examine homozygosity measures. Our analysis uses coalescent based methods.