Distance to the stochastic part of phylogenetic varieties

TL;DR

This paper investigates how the distance from observed data to algebraic varieties, especially their stochastic parts, can enhance phylogenetic tree reconstruction by integrating algebraic geometry and optimization techniques.

Contribution

It introduces methods to compute distances to algebraic varieties and their stochastic parts, improving phylogenetic inference accuracy.

Findings

Analytical distances computed for specific models

Use of nonlinear programming for distance optimization

Application of numerical algebraic geometry techniques

Abstract

Modelling the substitution of nucleotides along a phylogenetic tree is usually done by a hidden Markov process. This allows to define a distribution of characters at the leaves of the trees and one might be able to obtain polynomial relationships among the probabilities of different characters. The study of these polynomials and the geometry of the algebraic varieties defined by them can be used to reconstruct phylogenetic trees. However, not all points in these algebraic varieties have biological sense. In this paper, we explore the extent to which adding semi-algebraic conditions arising from the restriction to parameters with statistical meaning can improve existing methods of phylogenetic reconstruction. To this end, our aim is to compute the distance of data points to algebraic varieties and to the stochastic part of these varieties. Computing these distances involves optimization by nonlinear programming algorithms. We use analytical methods to find some of these distances for quartet trees evolving under the Kimura 3-parameter or the Jukes-Cantor models. Numerical algebraic geometry and computational algebra play also a fundamental role in this paper.