# Displacement-Optimized Tanglegrams for Trees and Networks

## TL;DR

This paper introduces a new method for visualizing phylogenetic trees and networks by minimizing taxon and network edge misalignment.

## Contribution

DO-tanglegrams is a novel approach that optimizes layouts for both trees and networks by minimizing taxon displacement and reticulate displacement.

## Key findings

- DO-tanglegrams outperformed existing methods like phytools::cophylo and NN-tanglegram on synthetic data.
- The algorithm handles unresolved nodes and missing taxa effectively.
- The method uses a combination of local search and simulated annealing to achieve optimization.

## Abstract

Phylogenetic trees and networks play a central role in biology, bioinformatics, and mathematical biology, and producing clear, informative visualizations of them is an important task. Tanglegrams, which display two phylogenies side by side with lines connecting shared taxa, are widely used for comparing evolutionary histories, host–parasite associations, and horizontal gene transfer. Existing layout algorithms have largely focused on trees and on minimizing the number of intertaxon edge crossings. We introduce displacement-optimized tanglegrams (DO-tanglegrams), a new approach that applies equally to trees and rooted phylogenetic networks. Our method explicitly minimizes taxon displacement—the vertical misalignment of corresponding taxa across the two sides—and reticulate displacement—the vertical distance spanned by reticulation edges within a network. We formalize one-sided and two-sided optimization problems, show that exact minimization is computationally intractable, and propose a heuristic that combines exhaustive local search with simulated annealing. The algorithm naturally accommodates unresolved nodes (multifurcations or multicombinations) and missing taxa. We have implemented the DO-tanglegram algorithm in SplitsTree. We compare our implementation against the phytools::cophylo R-function on a collection of synthetic trees, and against the NN-tanglegram algorithm in Dendroscope on a collection of synthetic networks. The results indicate that DO-tanglegram performs significantly better than cophylo on trees and then NN-tanglegram on networks.