On the Local Linear Rate of Consensus on the Stiefel Manifold

TL;DR

This paper analyzes the convergence of a Riemannian gradient method for consensus problems on the Stiefel manifold, establishing local linear convergence rates that match Euclidean space rates.

Contribution

It introduces the Distributed Riemannian Consensus on Stiefel Manifold (DRCS) and proves its local linear convergence rate matches the Euclidean case, a novel result.

Findings

DRCS achieves local linear convergence to global consensus.

The convergence rate asymptotically matches the second largest singular value of the communication matrix.

First work to show equality of convergence rates between Riemannian and Euclidean consensus algorithms.

Abstract

We study the convergence properties of Riemannian gradient method for solving the consensus problem (for an undirected connected graph) over the Stiefel manifold. The Stiefel manifold is a non-convex set and the standard notion of averaging in the Euclidean space does not work for this problem. We propose Distributed Riemannian Consensus on Stiefel Manifold (DRCS) and prove that it enjoys a local linear convergence rate to global consensus. More importantly, this local rate asymptotically scales with the second largest singular value of the communication matrix, which is on par with the well-known rate in the Euclidean space. To the best of our knowledge, this is the first work showing the equality of the two rates. The main technical challenges include (i) developing a Riemannian restricted secant inequality for convergence analysis, and (ii) to identify the conditions (e.g., suitable step-size and initialization) under which the algorithm always stays in the local region.