Protocols for Learning Classifiers on Distributed Data

TL;DR

This paper introduces communication-efficient protocols for learning classifiers on distributed datasets, leveraging sampling and active communication techniques to reduce data transfer while maintaining low approximation error.

Contribution

It proposes novel sampling-based and two-way communication protocols that significantly improve efficiency over traditional one-way methods for distributed classifier learning.

Findings

Protocols achieve exponential speed-up over one-way communication methods.

Techniques are inspired by active learning but focus on inter-node communication.

Methods are effective for noiseless data distributed across multiple nodes.

Abstract

We consider the problem of learning classifiers for labeled data that has been distributed across several nodes. Our goal is to find a single classifier, with small approximation error, across all datasets while minimizing the communication between nodes. This setting models real-world communication bottlenecks in the processing of massive distributed datasets. We present several very general sampling-based solutions as well as some two-way protocols which have a provable exponential speed-up over any one-way protocol. We focus on core problems for noiseless data distributed across two or more nodes. The techniques we introduce are reminiscent of active learning, but rather than actively probing labels, nodes actively communicate with each other, each node simultaneously learning the important data from another node.