Decentralized Online Riemannian Optimization with Dynamic Environments

TL;DR

This paper introduces a decentralized online Riemannian optimization algorithm on Hadamard manifolds, providing theoretical guarantees and practical simplifications, with experiments validating its effectiveness in nonstationary environments.

Contribution

It presents the first decentralized online Riemannian optimization algorithm with dynamic regret bounds and simplified consensus steps, advancing nonstationary decentralized optimization methods.

Findings

Achieves linear variance reduction in consensus steps.

Proves dynamic regret bounds of order ${ m O}(rac{ oot{ }{T(1+P_T)}}{ oot{ }{1-\sigma_2(W)}})$.

Validated effectiveness through experiments on hyperbolic spaces and SPD matrices.

Abstract

This paper develops the first decentralized online Riemannian optimization algorithm on Hadamard manifolds. Our algorithm, the decentralized projected Riemannian gradient descent, iteratively performs local updates using projected Riemannian gradient descent and a consensus step via weighted Frechet mean. Theoretically, we establish linear variance reduction for the consensus step. Building on this, we prove a dynamic regret bound of order ${\cal O}(\sqrt{T(1+P_T)}/\sqrt{(1-\sigma_2(W))})$, where $T$ is the time horizon, $P_T$ represents the path variation measuring nonstationarity, and $\sigma_2(W)$ measures the network connectivity. The weighted Frechet mean in our algorithm incurs a minimization problem, which can be computationally expensive. To further alleviate this cost, we propose a simplified consensus step with a closed-form, replacing the weighted Frechet mean. We then establish linear variance reduction for this alternative and prove that the decentralized algorithm, even with this simple consensus step, achieves the same dynamic regret bound. Finally, we validate our approach with experiments on nonstationary decentralized Frechet mean computation over hyperbolic spaces and the space of symmetric positive definite matrices, demonstrating the effectiveness of our methods.