Performance of a Distributed Stochastic Approximation Algorithm

TL;DR

This paper analyzes a distributed stochastic approximation algorithm combining local updates and gossip averaging, proving convergence and a Central Limit Theorem under weak assumptions, with applications in decentralized estimation and control.

Contribution

It introduces a convergence proof and second-order analysis for a distributed stochastic approximation algorithm with gossip steps, under minimal assumptions.

Findings

Convergence of estimates to consensus is established.

A Central Limit Theorem describes the asymptotic distribution.

Polyak averaging improves the algorithm's performance.

Abstract

In this paper, a distributed stochastic approximation algorithm is studied. Applications of such algorithms include decentralized estimation, optimization, control or computing. The algorithm consists in two steps: a local step, where each node in a network updates a local estimate using a stochastic approximation algorithm with decreasing step size, and a gossip step, where a node computes a local weighted average between its estimates and those of its neighbors. Convergence of the estimates toward a consensus is established under weak assumptions. The approach relies on two main ingredients: the existence of a Lyapunov function for the mean field in the agreement subspace, and a contraction property of the random matrices of weights in the subspace orthogonal to the agreement subspace. A second order analysis of the algorithm is also performed under the form of a Central Limit Theorem. The Polyak-averaged version of the algorithm is also considered.