Accelerated Optimization on Riemannian Manifolds via Discrete Constrained Variational Integrators

TL;DR

This paper develops time-adaptive Hamiltonian variational integrators for accelerated optimization on Riemannian manifolds, improving stability and efficiency by respecting geometric invariances and constraints.

Contribution

It introduces a method to incorporate holonomic constraints into discrete variational integrators for Riemannian manifolds, enhancing optimization algorithms' robustness.

Findings

Algorithms effectively solve eigenvalue problems on spheres.

Performance tested on Procrustes problems on Stiefel manifold.

Enhanced stability and efficiency over traditional methods.

Abstract

A variational formulation for accelerated optimization on normed vector spaces was recently introduced in Wibisono et al., and later generalized to the Riemannian manifold setting in Duruisseaux and Leok. This variational framework was exploited on normed vector spaces in Duruisseaux et al. using time-adaptive geometric integrators to design efficient explicit algorithms for symplectic accelerated optimization, and it was observed that geometric discretizations which respect the time-rescaling invariance and symplecticity of the Lagrangian and Hamiltonian flows were substantially less prone to stability issues, and were therefore more robust, reliable, and computationally efficient. As such, it is natural to develop time-adaptive Hamiltonian variational integrators for accelerated optimization on Riemannian manifolds. In this paper, we consider the case of Riemannian manifolds embedded in a Euclidean space that can be characterized as the level set of a submersion. We will explore how holonomic constraints can be incorporated in discrete variational integrators to constrain the numerical discretization of the Riemannian Hamiltonian system to the Riemannian manifold, and we will test the performance of the resulting algorithms by solving eigenvalue and Procrustes problems formulated as optimization problems on the unit sphere and Stiefel manifold.