Regularized Classification-Aware Quantization

TL;DR

This paper introduces a novel quantization method tailored for binary classification that balances reconstruction error and classification loss, ensuring robust performance on unseen data and faster training compared to existing approaches.

Contribution

It proposes a new class of algorithms that incorporate classification-aware regularization into quantization, improving generalization and efficiency for classification tasks.

Findings

Performs well on synthetic datasets like mixture and Gaussian data.

Achieves better generalization error compared to benchmark schemes.

Faster training complexity proportional to dataset size squared.

Abstract

Traditionally, quantization is designed to minimize the reconstruction error of a data source. When considering downstream classification tasks, other measures of distortion can be of interest; such as the 0-1 classification loss. Furthermore, it is desirable that the performance of these quantizers not deteriorate once they are deployed into production, as relearning the scheme online is not always possible. In this work, we present a class of algorithms that learn distributed quantization schemes for binary classification tasks. Our method performs well on unseen data, and is faster than previous methods proportional to a quadratic term of the dataset size. It works by regularizing the 0-1 loss with the reconstruction error. We present experiments on synthetic mixture and bivariate Gaussian data and compare training, testing, and generalization errors with a family of benchmark quantization schemes from the literature. Our method is called Regularized Classification-Aware Quantization.