Asymptotically Efficient Distributed Estimation With Exponential Family Statistics

TL;DR

This paper introduces a distributed estimation algorithm for multi-agent networks with exponential family data, achieving asymptotic efficiency comparable to centralized methods under certain network conditions.

Contribution

It proposes a novel distributed estimator that is asymptotically efficient and suitable for exponential family statistics, advancing the theory of distributed stochastic approximation.

Findings

Estimator achieves consistency under global observability and mean connectivity.

Estimates' asymptotic covariance matches the centralized Fisher information inverse.

The approach extends the theory of non-Markovian mixed timescale stochastic recursions.

Abstract

The paper studies the problem of distributed parameter estimation in multi-agent networks with exponential family observation statistics. A certainty-equivalence type distributed estimator of the consensus + innovations form is proposed in which, at each each observation sampling epoch agents update their local parameter estimates by appropriately combining the data received from their neighbors and the locally sensed new information (innovation). Under global observability of the networked sensing model, i.e., the ability to distinguish between different instances of the parameter value based on the joint observation statistics, and mean connectivity of the inter-agent communication network, the proposed estimator is shown to yield consistent parameter estimates at each network agent. Further, it is shown that the distributed estimator is asymptotically efficient, in that, the asymptotic covariances of the agent estimates coincide with that of the optimal centralized estimator, i.e., the inverse of the centralized Fisher information rate. From a technical viewpoint, the proposed distributed estimator leads to non-Markovian mixed timescale stochastic recursions and the analytical methods developed in the paper contribute to the general theory of distributed stochastic approximation.