Fast, asymptotically efficient, recursive estimation in a Riemannian manifold

TL;DR

This paper develops a Riemannian stochastic optimization algorithm for recursive statistical parameter estimation, achieving fast convergence and asymptotic efficiency without convexity assumptions, and demonstrates its effectiveness through numerical examples.

Contribution

It introduces the first Riemannian recursive estimation method that attains optimal asymptotic efficiency and fast convergence without requiring convexity of the divergence function.

Findings

Achieves a fast non-asymptotic convergence rate.

Proves asymptotic normality of recursive estimates.

Attains asymptotic efficiency using Fisher information metric.

Abstract

Stochastic optimisation in Riemannian manifolds, especially the Riemannian stochastic gradient method, has attracted much recent attention. The present work applies stochastic optimisation to the task of recursive estimation of a statistical parameter which belongs to a Riemannian manifold. Roughly, this task amounts to stochastic minimisation of a statistical divergence function. The following problem is considered : how to obtain fast, asymptotically efficient, recursive estimates, using a Riemannian stochastic optimisation algorithm with decreasing step sizes? In solving this problem, several original results are introduced. First, without any convexity assumptions on the divergence function, it is proved that, with an adequate choice of step sizes, the algorithm computes recursive estimates which achieve a fast non-asymptotic rate of convergence. Second, the asymptotic normality of these recursive estimates is proved, by employing a novel linearisation technique. Third, it is proved that, when the Fisher information metric is used to guide the algorithm, these recursive estimates achieve an optimal asymptotic rate of convergence, in the sense that they become asymptotically efficient. These results, while relatively familiar in the Euclidean context, are here formulated and proved for the first time, in the Riemannian context. In addition, they are illustrated with a numerical application to the recursive estimation of elliptically contoured distributions.