Distributed Nonparametric Estimation: from Sparse to Dense Samples per Terminal

TL;DR

This paper characterizes the optimal rates for distributed nonparametric function estimation under communication constraints, identifying phase transitions as the number of samples per terminal varies from sparse to dense, and proposes a layered estimation protocol.

Contribution

It fully solves the open problem of determining minimax optimal rates across all regimes of samples per terminal, introducing a new layered protocol and proving its optimality.

Findings

Identified phase transitions in optimal rates as samples per terminal change.

Designed a layered estimation protocol leveraging parametric density estimation techniques.

Established the optimality of the protocol using information-theoretic methods.

Abstract

Consider the communication-constrained problem of nonparametric function estimation, in which each distributed terminal holds multiple i.i.d. samples. Under certain regularity assumptions, we characterize the minimax optimal rates for all regimes, and identify phase transitions of the optimal rates as the samples per terminal vary from sparse to dense. This fully solves the problem left open by previous works, whose scopes are limited to regimes with either dense samples or a single sample per terminal. To achieve the optimal rates, we design a layered estimation protocol by exploiting protocols for the parametric density estimation problem. We show the optimality of the protocol using information-theoretic methods and strong data processing inequalities, and incorporating the classic balls and bins model. The optimal rates are immediate for various special cases such as density estimation, Gaussian, binary, Poisson and heteroskedastic regression models.