Maximum likelihood evidence for Neandertal admixture in Eurasian populations from three genomes

TL;DR

This paper develops a maximum likelihood method to distinguish between ancestral structure and Neandertal admixture in Eurasian genomes, providing formal statistical support for interbreeding events.

Contribution

It introduces a novel likelihood-based approach to differentiate admixture from ancestral structure using genomic data, applied to Neandertal and modern human genomes.

Findings

Strong support for Neandertal admixture into Eurasian populations.

Estimated admixture rate between 3.4% and 7.9%.

Rejection of ancestral population structure as sole explanation.

Abstract

Although there has been much interest in estimating divergence and admixture from genomic data, it has proven difficult to distinguish gene flow after divergence from alternative histories involving structure in the ancestral population. The lack of a formal test to distinguish these scenarios has sparked recent controversy about the possibility of interbreeding between Neandertals and modern humans in Eurasia. We derive the probability of mutational configurations in non-recombining sequence blocks under alternative histories of divergence with admixture and ancestral structure. Dividing the genome into short blocks makes it possible to compute maximum likelihood estimates of parameters under both models. We apply this method to triplets of human Neandertal genomes and quantify the relative support for models of long-term population structure in the ancestral African popuation and admixture from Neandertals into Eurasian populations after their expansion out of Africa. Our analysis allows us -- for the first time -- to formally reject a history of ancestral population structure and instead reveals strong support for admixture from Neandertals into Eurasian populations at a higher rate (3.4%-7.9%) than suggested previously.