Ternary and Binary Quantization for Improved Classification

TL;DR

This paper explores how combining random projection with ternary or binary quantization can reduce data complexity while maintaining or improving classification accuracy, especially with sparse features.

Contribution

It introduces a methodology of applying sparse random projections followed by ternary or binary quantization, demonstrating its effectiveness in classification tasks.

Findings

Quantization can maintain or improve classification accuracy with sparse features.

Sparse random projections preserve quantization benefits.

Extensive experiments validate the proposed approach.

Abstract

Dimension reduction and data quantization are two important methods for reducing data complexity. In the paper, we study the methodology of first reducing data dimension by random projection and then quantizing the projections to ternary or binary codes, which has been widely applied in classification. Usually, the quantization will seriously degrade the accuracy of classification due to high quantization errors. Interestingly, however, we observe that the quantization could provide comparable and often superior accuracy, as the data to be quantized are sparse features generated with common filters. Furthermore, this quantization property could be maintained in the random projections of sparse features, if both the features and random projection matrices are sufficiently sparse. By conducting extensive experiments, we validate and analyze this intriguing property.