A lattice structure for ancestral configurations arising from the relationship between gene trees and species trees

TL;DR

This paper introduces a lattice structure on ancestral configurations derived from gene and species trees, providing new methods to enumerate and analyze ancestral configurations and labeled histories in phylogenetics.

Contribution

It develops a lattice framework for ancestral configurations, relates it to labeled histories, and extends the approach to nonmatching gene and species tree pairs.

Findings

Established a bijection between paths on ancestral configuration digraphs and labeled histories.

Derived closed-form formulas for counting labeled histories in specific tree families.

Characterized pairs of gene and species trees with maximal ancestral configurations.

Abstract

To a given gene tree topology $G$ and species tree topology $S$ with leaves labeled bijectively from a fixed set $X$, one can associate a set of ancestral configurations, each of which encodes a set of gene lineages that can be found at a given node of the species tree. We introduce a lattice structure on ancestral configurations, studying the directed graphs that provide graphical representations of lattices of ancestral configurations. For a matching gene tree topology and species tree topology $G=S$, we present a method for defining the digraph of ancestral configurations from the tree topology by using iterated cartesian products of graphs. We show that a specific set of paths on the digraph of ancestral configurations is in bijection with the set of labeled histories -- a well-known phylogenetic object that enumerates possible temporal orderings of the coalescences of a tree. For each of a series of tree families, we obtain closed-form expressions for the number of labeled histories by using this bijection to count paths on associated digraphs. Finally, we prove that our lattice construction extends to nonmatching tree pairs, and we use it to characterize pairs $(G,S)$ having the maximal number of ancestral configurations for a fixed $G$. We discuss how the construction provides new methods for performing enumerations of combinatorial aspects of gene and species trees.