Polyhedral computational geometry for averaging metric phylogenetic trees

TL;DR

This paper develops computational geometry tools for calculating the Frechet mean and variance of probability distributions on phylogenetic tree space, enabling more efficient analysis of evolutionary data.

Contribution

It introduces a polyhedral subdivision approach to analyze geodesics and variance functions, and proposes iterative algorithms for computing Frechet means in tree space.

Findings

Polyhedral subdivision determines geodesic combinatorics in tree space.

Variance function is smooth within each cell of the subdivision.

Two iterative methods reliably converge to the Frechet mean.

Abstract

This paper investigates the computational geometry relevant to calculations of the Frechet mean and variance for probability distributions on the phylogenetic tree space of Billera, Holmes and Vogtmann, using the theory of probability measures on spaces of nonpositive curvature developed by Sturm. We show that the combinatorics of geodesics with a specified fixed endpoint in tree space are determined by the location of the varying endpoint in a certain polyhedral subdivision of tree space. The variance function associated to a finite subset of tree space has a fixed $C^\infty$ algebraic formula within each cell of the corresponding subdivision, and is continuously differentiable in the interior of each orthant of tree space. We use this subdivision to establish two iterative methods for producing sequences that converge to the Frechet mean: one based on Sturm's Law of Large Numbers, and another based on descent algorithms for finding optima of smooth functions on convex polyhedra. We present properties and biological applications of Frechet means and extend our main results to more general globally nonpositively curved spaces composed of Euclidean orthants.