Visualizing Multispecies Coalescent Trees: Drawing Gene Trees Inside Species Trees

TL;DR

This paper presents methods for visualizing multispecies coalescent trees by embedding multiple gene trees within a species tree diagram, focusing on minimizing crossings for clarity and providing heuristics and ILP solutions.

Contribution

It introduces novel visualization techniques for multispecies coalescent trees, including heuristics and an ILP formulation for optimal layouts.

Findings

Planar instances can be recognized in linear time

The problem is NP-hard in general

Heuristics perform well and ILP solutions are efficient

Abstract

We consider the problem of drawing multiple gene trees inside a single species tree in order to visualize multispecies coalescent trees. Specifically, the drawing of the species tree fills a rectangle in which each of its edges is represented by a smaller rectangle, and the gene trees are drawn as rectangular cladograms (that is, orthogonally and downward, with one bend per edge) inside the drawing of the species tree. As an alternative, we also consider a style where the widths of the edges of the species tree are proportional to given effective population sizes. In order to obtain readable visualizations, our aim is to minimize the number of crossings between edges of the gene trees in such drawings. We show that planar instances can be recognized in linear time and that the general problem is NP-hard. Therefore, we introduce two heuristics and give an integer linear programming (ILP) formulation that provides us with exact solutions in exponential time. We use the ILP to measure the quality of the heuristics on real-world instances. The heuristics yield surprisingly good solutions, and the ILP runs surprisingly fast.