Loopless Proximal Riemannian Gradient EXTRA for Distributed Optimization on Compact Manifolds

TL;DR

This paper introduces PR-EXTRA, a novel distributed optimization algorithm for compact Riemannian manifolds that requires only one communication round per iteration and guarantees convergence to stationary points.

Contribution

It extends the proximal gradient EXTRA algorithm to Riemannian manifolds, enabling efficient distributed composite optimization with theoretical convergence guarantees.

Findings

Achieves sublinear convergence rate of O(1/K)

Requires only a single communication round per iteration

Demonstrates effectiveness through numerical experiments

Abstract

Distributed optimization has gained substantial interest in recent years due to its wide applications in machine learning. However, most of existing algorithms are designed for Euclidean spaces, leaving composite optimization on Riemannian manifolds largely unexplored. To bridge this gap, we propose the proximal Riemannian gradient EXTRA algorithm (PR-EXTRA) to solve distributed composite optimization problem with nonsmooth regularizer over compact manifolds. In each iteration, PR-EXTRA requires only a single round communication, coupled with local gradient evaluations and proximal mappings. Furthermore, a manifold projection operator is integrated to ensure the feasibility of all iterates throughout the optimization process. Theoretical analysis shows that with a constant stepsize, PR-EXTRA achieves a sublinear convergence rate of $\mathcal{O}(1/K)$ to a stationary point, matching the proximal gradient EXTRA algorithm in Euclidean spaces. Numerical experiments show the effectiveness of the proposed algorithm.