Computing Euclidean distance and maximum likelihood retraction maps for constrained optimization

TL;DR

This paper develops algorithms for computing retraction maps on implicitly-defined manifolds, including those based on maximum likelihood, with convergence proofs and applications to statistical models.

Contribution

It introduces a homotopy continuation method for Euclidean distance retractions on implicit manifolds and proves the second-order accuracy of MLE-based retractions.

Findings

Homotopy continuation effectively computes retractions on implicit manifolds.

MLE-based retraction maps are second-order and approximate geodesics.

The methods apply to statistical models with Fisher metric.

Abstract

Riemannian optimization uses local methods to solve optimization problems whose constraint set is a smooth manifold. A linear step along some descent direction usually leaves the constraints, and hence retraction maps are used to approximate the exponential map and return to the manifold. For many common matrix manifolds, retraction maps are available, with more or less explicit formulas. For implicitly-defined manifolds, suitable retraction maps are difficult to compute. We therefore develop an algorithm which uses homotopy continuation to compute the Euclidean distance retraction for any implicitly-defined submanifold of R^n, and prove convergence results. We also consider statistical models as Riemannian submanifolds of the probability simplex with the Fisher metric. Replacing Euclidean distance with maximum likelihood results in a map which we prove is a retraction. In fact, we prove the retraction is second-order; with the Levi-Civita connection associated to the Fisher metric, it approximates geodesics to second-order accuracy.