Distributed Parameter Estimation via Pseudo-likelihood

TL;DR

This paper introduces a distributed learning method using pseudo-likelihood components that is computationally efficient, communication-friendly, and statistically competitive, suitable for sensor networks.

Contribution

It proposes a general framework for distributed parameter estimation combining local pseudo-likelihood estimators with simple aggregation methods, supported by theoretical and experimental validation.

Findings

Linear combination with second-order info is statistically competitive.

Algorithms have low communication and computational costs.

Methods exhibit 'any-time' behavior, enabling flexible deployment.

Abstract

Estimating statistical models within sensor networks requires distributed algorithms, in which both data and computation are distributed across the nodes of the network. We propose a general approach for distributed learning based on combining local estimators defined by pseudo-likelihood components, encompassing a number of combination methods, and provide both theoretical and experimental analysis. We show that simple linear combination or max-voting methods, when combined with second-order information, are statistically competitive with more advanced and costly joint optimization. Our algorithms have many attractive properties including low communication and computational cost and "any-time" behavior.