Distributed Banach-Picard Iteration: Application to Distributed EM and Distributed PCA

TL;DR

This paper extends the distributed Banach-Picard iteration to develop distributed algorithms for PCA and EM, providing local linear convergence guarantees for these algorithms in sensor networks.

Contribution

It demonstrates that PCA and EM algorithms can be formulated as locally contractive maps, enabling distributed solutions with convergence guarantees.

Findings

Distributed PCA and EM algorithms with proven local linear convergence.

PCA and EM are shown to be expressible as averages of local maps.

Probabilistic convergence guarantees for the EM algorithm in noisy sensor networks.

Abstract

In recent work, we proposed a distributed Banach-Picard iteration (DBPI) that allows a set of agents, linked by a communication network, to find a fixed point of a locally contractive (LC) map that is the average of individual maps held by said agents. In this work, we build upon the DBPI and its local linear convergence (LLC) guarantees to make several contributions. We show that Sanger's algorithm for principal component analysis (PCA) corresponds to the iteration of an LC map that can be written as the average of local maps, each map known to each agent holding a subset of the data. Similarly, we show that a variant of the expectation-maximization (EM) algorithm for parameter estimation from noisy and faulty measurements in a sensor network can be written as the iteration of an LC map that is the average of local maps, each available at just one node. Consequently, via the DBPI, we derive two distributed algorithms - distributed EM and distributed PCA - whose LLC guarantees follow from those that we proved for the DBPI. The verification of the LC condition for EM is challenging, as the underlying operator depends on random samples, thus the LC condition is of probabilistic nature.