Finite Sample Guarantees for Distributed Online Parameter Estimation with Communication Costs

TL;DR

This paper presents a distributed online parameter estimation algorithm with finite-sample error guarantees, balancing estimation accuracy and communication costs, and determining optimal stopping times for communication.

Contribution

It introduces a non-asymptotic analysis for distributed online estimation, providing finite-sample error bounds and communication trade-offs, which was lacking in prior asymptotic-focused work.

Findings

Finite-sample error bounds for distributed online estimation.

Trade-off characterization between estimation accuracy and communication costs.

Method to determine optimal communication stopping time.

Abstract

We study the problem of estimating an unknown parameter in a distributed and online manner. Existing work on distributed online learning typically either focuses on asymptotic analysis, or provides bounds on regret. However, these results may not directly translate into bounds on the error of the learned model after a finite number of time-steps. In this paper, we propose a distributed online estimation algorithm which enables each agent in a network to improve its estimation accuracy by communicating with neighbors. We provide non-asymptotic bounds on the estimation error, leveraging the statistical properties of the underlying model. Our analysis demonstrates a trade-off between estimation error and communication costs. Further, our analysis allows us to determine a time at which the communication can be stopped (due to the costs associated with communications), while meeting a desired estimation accuracy. We also provide a numerical example to validate our results.