Communication-Efficient Distributed Statistical Inference

TL;DR

This paper introduces a communication-efficient surrogate likelihood framework for distributed statistical inference, enhancing estimation accuracy, confidence interval construction, and Bayesian inference with reduced communication costs.

Contribution

The paper proposes the CSL framework that improves distributed estimation, provides minimax-optimal high-dimensional estimators, and enhances Bayesian inference efficiency.

Findings

CSL outperforms naive averaging in low-dimensional estimation.

CSL achieves minimax-optimal rates with low communication in high-dimensional settings.

CSL significantly accelerates MCMC-based Bayesian inference.

Abstract

We present a Communication-efficient Surrogate Likelihood (CSL) framework for solving distributed statistical inference problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation and Bayesian inference. For low-dimensional estimation, CSL provably improves upon naive averaging schemes and facilitates the construction of confidence intervals. For high-dimensional regularized estimation, CSL leads to a minimax-optimal estimator with controlled communication cost. For Bayesian inference, CSL can be used to form a communication-efficient quasi-posterior distribution that converges to the true posterior. This quasi-posterior procedure significantly improves the computational efficiency of MCMC algorithms even in a non-distributed setting. We present both theoretical analysis and experiments to explore the properties of the CSL approximation.