Iso-Riemannian Optimization on Learned Data Manifolds

TL;DR

This paper introduces iso-Riemannian optimization, a new geometric framework for optimizing on learned data manifolds, enabling effective algorithms and convergence guarantees in high-dimensional machine learning tasks.

Contribution

It develops an iso-Riemannian framework with new notions of convexity and monotonicity, and proposes an algorithm with convergence analysis tailored for learned data manifolds.

Findings

Iso-Riemannian descent algorithm converges under new convexity notions.

Improved clustering and interpretable barycentres on datasets like MNIST.

Effective solutions to inverse problems in high-dimensional settings.

Abstract

High-dimensional data with intrinsic low-dimensional structure is ubiquitous in machine learning and data science. While various approaches allow one to learn a data manifold with a Riemannian structure from finite samples, performing downstream tasks such as optimization directly on these learned manifolds remains challenging. In particular, Euclidean convex functions cannot be assumed to be geodesically convex, and the associated Riemannian gradient fields are generally not monotone in the classical Riemannian sense. As a result, existing Riemannian optimization theory neither identifies a canonical vector field to use in first-order schemes nor guarantees their convergence in this setting. To address this, we introduce notions of convexity, monotonicity, and Lipschitz continuity induced by a connection different from the Levi-Civita connection, namely the recently proposed iso-connection. Within this iso-Riemannian framework, we propose an iso-Riemannian descent algorithm and provide a detailed convergence analysis. We then show, for several downstream tasks - including iso-Riemannian barycentre computation and the optimization of Euclidean convex functions over learned data manifolds - that iso-convexity, iso-monotonicity, and iso-Lipschitz continuity form the right set of assumptions to reconcile learned geometry with Euclidean convexity. Experiments on synthetic and real datasets, including MNIST, endowed with a learned pullback structure, demonstrate that our approach yields interpretable barycentres, improved clustering, and provably efficient solutions to inverse problems, even in high-dimensional settings. Taken together, these results show that iso-Riemannian optimization provides a natural geometric framework for designing and analyzing algorithms on learned data manifolds.