Intrinsic Decentralized Stochastic Riemannian Optimization on Manifolds with Bounded Sectional Curvature

TL;DR

This paper introduces a decentralized stochastic Riemannian gradient method with diminishing step sizes for optimization on manifolds with bounded sectional curvature, achieving the first non-asymptotic optimality guarantees in this setting.

Contribution

It provides the first exact, non-asymptotic convergence guarantees for intrinsic decentralized stochastic Riemannian optimization with diminishing step sizes.

Findings

Achieves $O(1/T)$ consensus error bound.

Obtains $O(rac{ ext{log} T}{ oot{2} ext{T}})$ optimality gap.

Demonstrates improved practical performance on distributed PCA.

Abstract

Decentralized optimization on Riemannian manifolds is foundational for many modern machine learning and signal processing applications in which data are non-Euclidean and generated and processed in a distributed manner. Although intrinsic Riemannian methods exploit manifold geometry without relying on Euclidean embeddings, existing decentralized Riemannian optimization algorithms typically use constant step sizes and therefore converge only to a neighborhood of steady-state error. In this paper, we study the decentralized stochastic Riemannian gradient method in the diminishing step-size regime on manifolds with (possibly positive) bounded sectional curvature. We prove an $O(1/T)$ bound for the network consensus error and an $O(\log T/\sqrt{T})$ ergodic bound for the global optimality gap. To the best of our knowledge, this is the first exact, non-asymptotic optimality-gap guarantee for an intrinsic decentralized stochastic Riemannian method in the geodesically convex setting. Furthermore, the diminishing step-size schedule allows substantially larger initial gradient steps than fixed-step baselines, leading to better performance in practice. We illustrate this on the problem of distributed PCA over a Grassmann manifold.