On Distributed Quantization for Classification

TL;DR

This paper develops distributed feature quantization schemes optimized for classification tasks, significantly reducing communication costs without sacrificing accuracy, and provides theoretical and practical algorithms for their design.

Contribution

It introduces task-specific quantization methods for distributed classification, including NP-hardness proof, optimal solutions for special cases, and polynomial-time algorithms leveraging training data.

Findings

Over twofold reduction in communication bits for same accuracy

Proposed algorithms work for any number of features and classes

Quantization tailored to classification improves efficiency significantly

Abstract

We consider the problem of distributed feature quantization, where the goal is to enable a pretrained classifier at a central node to carry out its classification on features that are gathered from distributed nodes through communication constrained channels. We propose the design of distributed quantization schemes specifically tailored to the classification task: unlike quantization schemes that help the central node reconstruct the original signal as accurately as possible, our focus is not reconstruction accuracy, but instead correct classification. Our work does not make any apriori distributional assumptions on the data, but instead uses training data for the quantizer design. Our main contributions include: we prove NP-hardness of finding optimal quantizers in the general case; we design an optimal scheme for a special case; we propose quantization algorithms, that leverage discrete neural representations and training data, and can be designed in polynomial-time for any number of features, any number of classes, and arbitrary division of features across the distributed nodes. We find that tailoring the quantizers to the classification task can offer significant savings: as compared to alternatives, we can achieve more than a factor of two reduction in terms of the number of bits communicated, for the same classification accuracy.