Riemannian EXTRA: Communication-efficient decentralized optimization over compact submanifolds with data heterogeneity

TL;DR

This paper introduces REXTRA, a novel Riemannian decentralized optimization algorithm that achieves global sublinear convergence with reduced communication, effectively handling data heterogeneity over compact submanifolds.

Contribution

RETRA is the first Riemannian decentralized algorithm to attain global sublinear convergence with a single communication round per iteration under constant step size.

Findings

Achieves a convergence rate of O(1/k) matching the best-known results.

Supports larger step sizes compared to existing methods.

Reduces total communication by over 50% in experiments.

Abstract

We consider decentralized optimization over a compact Riemannian submanifold in a network of $n$ agents, where each agent holds a smooth, nonconvex local objective defined by its private data. The goal is to collaboratively minimize the sum of these local objective functions. In the presence of data heterogeneity across nodes, existing algorithms typically require communicating both local gradients and iterates to ensure exact convergence with constant step sizes. In this work, we propose REXTRA, a Riemannian extension of the EXTRA algorithm [Shi et al., SIOPT, 2015], to address this limitation. On the theoretical side, we leverage proximal smoothness to overcome the challenges of manifold nonconvexity and establish a global sublinear convergence rate of $\mathcal{O}(1/k)$, matching the best-known results. To our knowledge, REXTRA is the first algorithm to achieve a global sublinear convergence rate under a constant step size while requiring only a single round of local iterate communication per iteration. Numerical experiments show that REXTRA achieves superior performance compared to state-of-the-art methods, while supporting larger step sizes and reducing total communication by over 50\%.