Distributed Estimation and Learning over Heterogeneous Networks

TL;DR

This paper develops distributed algorithms for estimation and learning in heterogeneous networks, enabling agents to efficiently aggregate data over time and across nodes without centralized coordination, with proven convergence and finite-time guarantees.

Contribution

It introduces novel distributed aggregation schemes for heterogeneous networks, handling data heterogeneity and intermittence, with rigorous convergence and performance analysis.

Findings

Algorithms converge to globally efficient estimators.

Finite-time performance guarantees are established.

Methods effectively handle data heterogeneity and intermittent streams.

Abstract

We consider several estimation and learning problems that networked agents face when making decisions given their uncertainty about an unknown variable. Our methods are designed to efficiently deal with heterogeneity in both size and quality of the observed data, as well as heterogeneity over time (intermittence). The goal of the studied aggregation schemes is to efficiently combine the observed data that is spread over time and across several network nodes, accounting for all the network heterogeneities. Moreover, we require no form of coordination beyond the local neighborhood of every network agent or sensor node. The three problems that we consider are (i) maximum likelihood estimation of the unknown given initial data sets, (ii) learning the true model parameter from streams of data that the agents receive intermittently over time, and (iii) minimum variance estimation of a complete sufficient statistic from several data points that the networked agents collect over time. In each case we rely on an aggregation scheme to combine the observations of all agents; moreover, when the agents receive streams of data over time, we modify the update rules to accommodate the most recent observations. In every case, we demonstrate the efficiency of our algorithms by proving convergence to the globally efficient estimators given the observations of all agents. We supplement these results by investigating the rate of convergence and providing finite-time performance guarantees.