Consistency in Models for Distributed Learning under Communication Constraints

TL;DR

This paper investigates whether universal consistency guarantees in statistical learning, like Stone's Theorem, extend to distributed models with communication constraints, focusing on binary classification and regression.

Contribution

It introduces and analyzes distributed learning models with communication limits, examining the conditions for universal consistency in these constrained environments.

Findings

Universal consistency may not hold under certain communication constraints.

The paper identifies conditions where consistent ensemble learning is achievable.

It highlights new challenges in extending classical statistical guarantees to distributed settings.

Abstract

Motivated by sensor networks and other distributed settings, several models for distributed learning are presented. The models differ from classical works in statistical pattern recognition by allocating observations of an independent and identically distributed (i.i.d.) sampling process amongst members of a network of simple learning agents. The agents are limited in their ability to communicate to a central fusion center and thus, the amount of information available for use in classification or regression is constrained. For several basic communication models in both the binary classification and regression frameworks, we question the existence of agent decision rules and fusion rules that result in a universally consistent ensemble. The answers to this question present new issues to consider with regard to universal consistency. Insofar as these models present a useful picture of distributed scenarios, this paper addresses the issue of whether or not the guarantees provided by Stone's Theorem in centralized environments hold in distributed settings.