Gossip PCA

TL;DR

This paper explores a gossip algorithm-based method for computing leading eigenvectors of large data matrices, addressing challenges in distributed settings with limited communication.

Contribution

It introduces a novel gossip algorithm that generalizes consensus methods to compute eigenvectors without central coordination.

Findings

The algorithm converges under certain conditions.

It effectively approximates eigenvectors in distributed environments.

The analysis uses Markov chain theory for convergence guarantees.

Abstract

Eigenvectors of data matrices play an important role in many computational problems, ranging from signal processing to machine learning and control. For instance, algorithms that compute positions of the nodes of a wireless network on the basis of pairwise distance measurements require a few leading eigenvectors of the distances matrix. While eigenvector calculation is a standard topic in numerical linear algebra, it becomes challenging under severe communication or computation constraints, or in absence of central scheduling. In this paper we investigate the possibility of computing the leading eigenvectors of a large data matrix through gossip algorithms. The proposed algorithm amounts to iteratively multiplying a vector by independent random sparsification of the original matrix and averaging the resulting normalized vectors. This can be viewed as a generalization of gossip algorithms for consensus, but the resulting dynamics is significantly more intricate. Our analysis is based on controlling the convergence to stationarity of the associated Kesten-Furstenberg Markov chain.