Identifying species network features from gene tree quartets under the coalescent model

TL;DR

This paper demonstrates that key features of level-1 species networks can be identified from gene tree quartets under the coalescent model, advancing the theoretical foundation for network inference.

Contribution

It establishes the identifiability of certain network features from gene tree data and provides methods to compute quartet concordance factors for complex networks.

Findings

Identifiable features include cycles of size ≥4 and hybrid nodes in cycles of size ≥5.

Provides formulas to compute quartet concordance factors from simpler networks.

Explores conditions for detecting 3-cycles and 4-cycle hybrid nodes.

Abstract

We show that many topological features of level-1 species networks are identifiable from the distribution of the gene tree quartets under the network multi-species coalescent model. In particular, every cycle of size at least 4 and every hybrid node in a cycle of size at least 5 is identifiable. This is a step toward justifying the inference of such networks which was recently implemented by Sol\'is-Lemus and An\'e. We show additionally how to compute quartet concordance factors for a network in terms of simpler networks, and explore some circumstances in which cycles of size 3 and hybrid nodes in 4-cycles can be detected.