Evaluating Phylogenetic Comparative Methods under Reticulate Evolutionary Scenarios

TL;DR

This study evaluates how reticulate evolution, like hybridization, impacts the accuracy of phylogenetic comparative methods, revealing significant errors under complex evolutionary scenarios and offering guidance for researchers.

Contribution

It systematically assesses the performance of PCMs on phylogenetic networks with hybridization, highlighting factors that cause inaccuracies and providing recommendations.

Findings

Hybridization and rapid trait evolution increase estimation errors.

Tree-based PCMs often misinterpret complex evolutionary histories.

Factors like recent hybridization and transgressive events significantly affect accuracy.

Abstract

Phylogenetic comparative methods (PCMs) are widely used to study trait evolution. However, many evolutionary histories involve reticulate evolutionary scenarios, such as hybridization, that violate core assumptions of these methods. In this study, we evaluate how such violations affect the performance of PCMs. In particular, we focus on the ancestral character estimation, evolutionary rate estimation, and model selection. We simulate continuous trait evolution on various phylogenetic network topologies and assess the performance of PCMs that assume a bifurcating tree (i.e., major tree of the network) as the underlying model of evolution. We found that the performance of the tested PCMs was suboptimal. Using random forest, generalized linear models, and model-based clustering, we identified key factors contributing to these inaccuracies. Our results show that frequent and/or recent hybridization accompanied by one ore more transgressive events and rapidly evolving traits (i.e., high evolutionary rate) lead to significant estimation error, especially with respect to rate estimation and model choice. These factors substantially shift trait values away from tree-based model expectations, leading to overall increased error in parameter estimates. Our study demonstrates cases in which PCMs that rely on trees are likely to misinterpret biological histories and offers recommendations for researchers studying systems with complex evolutionary histories.