Communication Efficient Parallel Algorithms for Optimization on Manifolds

TL;DR

This paper introduces communication-efficient parallel algorithms for optimization on manifolds, extending distributed inference methods beyond Euclidean spaces, with proven convergence and demonstrated effectiveness on spherical data and matrix completion tasks.

Contribution

It generalizes parallel inference algorithms to manifold optimization, providing theoretical guarantees and practical performance improvements.

Findings

Algorithm is communication efficient and converges theoretically.

Effective in estimating Fréchet means on spherical data.

Performs well on low-rank matrix completion for Netflix data.

Abstract

The last decade has witnessed an explosion in the development of models, theory and computational algorithms for "big data" analysis. In particular, distributed computing has served as a natural and dominating paradigm for statistical inference. However, the existing literature on parallel inference almost exclusively focuses on Euclidean data and parameters. While this assumption is valid for many applications, it is increasingly more common to encounter problems where the data or the parameters lie on a non-Euclidean space, like a manifold for example. Our work aims to fill a critical gap in the literature by generalizing parallel inference algorithms to optimization on manifolds. We show that our proposed algorithm is both communication efficient and carries theoretical convergence guarantees. In addition, we demonstrate the performance of our algorithm to the estimation of Fr\'echet means on simulated spherical data and the low-rank matrix completion problem over Grassmann manifolds applied to the Netflix prize data set.