There are no caterpillars in a wicked forest

TL;DR

This paper proves that caterpillar gene tree topologies cannot be anomalous under the multispecies coalescent model, providing new insights into species tree inference in rapid speciation scenarios.

Contribution

It establishes a universal constraint that caterpillar gene trees are never anomalous, introducing the novel concept of population histories for proof.

Findings

Caterpillar gene trees are never anomalous.

Introduces the concept of population histories.

Provides a combinatorial proof for the constraint.

Abstract

Species trees represent the historical divergences of populations or species, while gene trees trace the ancestry of individual gene copies sampled within those populations. In cases involving rapid speciation, gene trees with topologies that differ from that of the species tree can be most probable under the standard multispecies coalescent model, making species tree inference more difficult. Such anomalous gene trees are not well understood except for some small cases. In this work, we establish one constraint that applies to trees of any size: gene trees with "caterpillar" topologies cannot be anomalous. The proof of this involves a new combinatorial object, called a population history, which keeps track of the number of coalescent events in each ancestral population.