Riemannian Optimistic Algorithms

TL;DR

This paper introduces novel Riemannian optimistic algorithms for online convex optimization, analyzing their regret and convergence properties, and demonstrating their effectiveness in zero-sum games and equilibrium seeking.

Contribution

The paper develops and analyzes new Riemannian optimistic algorithms that extend classical methods to curved spaces, with theoretical guarantees matching Euclidean results.

Findings

Algorithms achieve regret bounds comparable to Euclidean cases.

Proven last-iterate convergence for strongly convex-strongly concave problems.

Experimental results confirm theoretical predictions.

Abstract

In this paper, we consider Riemannian online convex optimization with dynamic regret. First, we propose two novel algorithms, namely the Riemannian Online Optimistic Gradient Descent (R-OOGD) and the Riemannian Adaptive Online Optimistic Gradient Descent (R-AOOGD), which combine the advantages of classical optimistic algorithms with the rich geometric properties of Riemannian manifolds. We analyze the dynamic regrets of the R-OOGD and R-AOOGD in terms of regularity of the sequence of cost functions and comparators. Next, we apply the R-OOGD to Riemannian zero-sum games, leading to the Riemannian Optimistic Gradient Descent Ascent algorithm (R-OGDA). We analyze the average iterate and best-iterate of the R-OGDA in seeking Nash equilibrium for a two-player, zero-sum, g-convex-concave games. We also prove the last-iterate convergence of the R-OGDA for g-strongly convex-strongly concave problems. Our theoretical analysis shows that all proposed algorithms achieve results in regret and convergence that match their counterparts in Euclidean spaces. Finally, we conduct several experiments to verify our theoretical findings.