Tanglegrams: a reduction tool for mathematical phylogenetics

TL;DR

This paper explores tanglegrams as combinatorial objects in phylogenetics, showing how they simplify the analysis of problems involving leaf-labeled trees and revealing their symmetries.

Contribution

It formalizes the isomorphism types of tanglegrams using double cosets and investigates their automorphisms, enhancing understanding of their combinatorial structure.

Findings

Tanglegrams can be understood via double cosets of the symmetric group.

Automorphisms of tanglegrams are characterized and studied.

Many phylogenetic problems factor through tanglegram analysis.

Abstract

Many discrete mathematics problems in phylogenetics are defined in terms of the relative labeling of pairs of leaf-labeled trees. These relative labelings are naturally formalized as tanglegrams, which have previously been an object of study in coevolutionary analysis. Although there has been considerable work on planar drawings of tanglegrams, they have not been fully explored as combinatorial objects until recently. In this paper, we describe how many discrete mathematical questions on trees "factor" through a problem on tanglegrams, and how understanding that factoring can simplify analysis. Depending on the problem, it may be useful to consider a unordered version of tanglegrams, and/or their unrooted counterparts. For all of these definitions, we show how the isomorphism types of tanglegrams can be understood in terms of double cosets of the symmetric group, and we investigate their automorphisms. Understanding tanglegrams better will isolate the distinct problems on leaf-labeled pairs of trees and reveal natural symmetries of spaces associated with such problems.