Enumeration of ancestral configurations for matching gene trees and species trees

TL;DR

This paper provides enumerative results on ancestral configurations in matching gene trees and species trees, revealing exponential growth patterns and their implications for computational biology.

Contribution

It introduces new enumerative results on ancestral configurations, highlighting their exponential growth and relation to tree shape in gene and species tree matching.

Findings

Number of ancestral configurations grows exponentially with taxa.

Maximal configurations occur for trees with specific shapes.

Mean and variance of configurations grow exponentially under uniform distribution.

Abstract

Given a gene tree and a species tree, ancestral configurations represent the combinatorially distinct sets of gene lineages that can reach a given node of the species tree. They have been introduced as a data structure for use in the recursive computation of the conditional probability under the multispecies coalescent model of a gene tree topology given a species tree, the cost of this computation being affected by the number of ancestral configurations of the gene tree in the species tree. For matching gene trees and species trees, we obtain enumerative results on ancestral configurations. We study ancestral configurations in balanced and unbalanced families of trees determined by a given seed tree, showing that for seed trees with more than one taxon, the number of ancestral configurations increases for both families exponentially in the number of taxa $n$. For fixed $n$, the maximal number of ancestral configurations tabulated at the species tree root node and the largest number of labeled histories possible for a labeled topology occur for trees with precisely the same unlabeled shape. For ancestral configurations at the root, the maximum increases with $k_0^n$, where $k_0 \approx 1.5028$ is a quadratic recurrence constant. Under a uniform distribution over the set of labeled trees of given size, the mean number of root ancestral configurations grows with $\sqrt{3/2}(4/3)^n$ and the variance with approximately $1.4048(1.8215)^n$. The results provide a contribution to the combinatorial study of gene trees and species trees.