Distinguishing Phylogenetic Level-2 Networks with Quartets and Inter-Taxon Quartet Distances

TL;DR

This paper develops a theoretically grounded, statistically consistent method for inferring complex semi-directed level-2 phylogenetic networks using quartet data, extending previous methods limited to simpler level-1 networks.

Contribution

It introduces a new inference framework for semi-directed level-2 networks, characterizes distinguishable features, and proves the circular decomposability of inter-taxon distances derived from quartets.

Findings

Established theoretical foundations for level-2 network inference

Proved inter-taxon distances are circular decomposable

Demonstrated identifiability across data types and models

Abstract

The inference of phylogenetic networks, which model complex evolutionary processes including hybridization and gene flow, remains a central challenge in evolutionary biology. Until now, statistically consistent inference methods have been limited to phylogenetic level-1 networks, which allow no interdependence between reticulate events. In this work, we establish the theoretical foundations for a statistically consistent inference method for a much broader class: semi-directed level-2 networks that are outer-labeled planar and galled. We precisely characterize the features of these networks that are distinguishable from the topologies of their displayed quartet trees. Moreover, we prove that an inter-taxon distance derived from these quartets is circular decomposable, enabling future robust inference of these networks from quartet data, such as concordance factors obtained from gene tree distributions under the Network Multispecies Coalescent model. Our results also have novel identifiability implications across different data types and evolutionary models, applying to any setting in which displayed quartets can be distinguished.