Stochastic gradient descent on Riemannian manifolds

TL;DR

This paper extends stochastic gradient descent to Riemannian manifolds, proving convergence and demonstrating applications including a new gossip algorithm for covariance matrices.

Contribution

It introduces a stochastic gradient descent method on Riemannian manifolds with convergence guarantees and practical applications like a novel covariance matrix gossip algorithm.

Findings

Proves convergence of Riemannian stochastic gradient descent.

Develops a new gossip algorithm for covariance matrices.

Demonstrates the algorithm's effectiveness through numerical tests.

Abstract

Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where the function is defined on a Riemannian manifold. We prove that, as in the Euclidian case, the gradient descent algorithm converges to a critical point of the cost function. The algorithm has numerous potential applications, and is illustrated here by four examples. In particular a novel gossip algorithm on the set of covariance matrices is derived and tested numerically.