Fourier transform inequalities for phylogenetic trees

TL;DR

This paper introduces new edge-parameter inequalities in Fourier coordinates that, combined with phylogenetic invariants, fully characterize valid site-pattern frequency vectors for phylogenetic trees.

Contribution

It provides the first complete set of inequalities in Fourier coordinates that define the semialgebraic sets of valid phylogenetic site-pattern frequencies.

Findings

Edge-parameter inequalities are necessary constraints on site-pattern vectors.

Two complete sets of inequalities are derived for group-based models.

These inequalities, combined with invariants, fully describe the phylogenetic parameter space.

Abstract

Phylogenetic invariants are not the only constraints on site-pattern frequency vectors for phylogenetic trees. A mutation matrix, by its definition, is the exponential of a matrix with non-negative off-diagonal entries; this positivity requirement implies non-trivial constraints on the site-pattern frequency vectors. We call these additional constraints ``edge-parameter inequalities.'' In this paper, we first motivate the edge-parameter inequalities by considering a pathological site-pattern frequency vector corresponding to a quartet tree with a negative internal edge. This site-pattern frequency vector nevertheless satisfies all of the constraints described up to now in the literature. We next describe two complete sets of edge-parameter inequalities for the group-based models; these constraints are square-free monomial inequalities in the Fourier transformed coordinates. These inequalities, along with the phylogenetic invariants, form a complete description of the set of site-pattern frequency vectors corresponding to \emph{bona fide} trees. Said in mathematical language, this paper explicitly presents two finite lists of inequalities in Fourier coordinates of the form ``monomial $\leq 1$,'' each list characterizing the phylogenetically relevant semialgebraic subsets of the phylogenetic varieties.