Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold

TL;DR

This paper introduces a decentralized Riemannian conjugate gradient method for optimizing functions on the Stiefel manifold in distributed networks, reducing computational complexity and achieving global convergence.

Contribution

It proposes the first decentralized Riemannian conjugate gradient algorithm that avoids complex geometric operations and guarantees convergence on the Stiefel manifold.

Findings

Achieves global convergence for decentralized optimization on the Stiefel manifold.

Reduces computational complexity by avoiding expensive geometric operations.

First to extend conjugate gradient methods to decentralized Riemannian settings.

Abstract

The conjugate gradient method is a crucial first-order optimization method that generally converges faster than the steepest descent method, and its computational cost is much lower than that of second-order methods. However, while various types of conjugate gradient methods have been studied in Euclidean spaces and on Riemannian manifolds, there is little study for those in distributed scenarios. This paper proposes a decentralized Riemannian conjugate gradient descent (DRCGD) method that aims at minimizing a global function over the Stiefel manifold. The optimization problem is distributed among a network of agents, where each agent is associated with a local function, and the communication between agents occurs over an undirected connected graph. Since the Stiefel manifold is a non-convex set, a global function is represented as a finite sum of possibly non-convex (but smooth) local functions. The proposed method is free from expensive Riemannian geometric operations such as retractions, exponential maps, and vector transports, thereby reducing the computational complexity required by each agent. To the best of our knowledge, DRCGD is the first decentralized Riemannian conjugate gradient algorithm to achieve global convergence over the Stiefel manifold.