Quantized Compressive K-Means

TL;DR

This paper extends compressive K-Means to include hardware-friendly periodic nonlinearities, especially 1-bit quantization, enabling resource-efficient data acquisition with minimal impact on clustering accuracy.

Contribution

It generalizes the CKM framework to a broad class of nonlinearities, including quantization, and demonstrates effective clustering with minimal resource use.

Findings

Quantized CKM performs comparably to standard CKM in clustering accuracy.

The method efficiently uses 1-bit quantization for data sketching.

Numerical experiments confirm minimal performance loss with resource-efficient signatures.

Abstract

The recent framework of compressive statistical learning aims at designing tractable learning algorithms that use only a heavily compressed representation-or sketch-of massive datasets. Compressive K-Means (CKM) is such a method: it estimates the centroids of data clusters from pooled, non-linear, random signatures of the learning examples. While this approach significantly reduces computational time on very large datasets, its digital implementation wastes acquisition resources because the learning examples are compressed only after the sensing stage. The present work generalizes the sketching procedure initially defined in Compressive K-Means to a large class of periodic nonlinearities including hardware-friendly implementations that compressively acquire entire datasets. This idea is exemplified in a Quantized Compressive K-Means procedure, a variant of CKM that leverages 1-bit universal quantization (i.e. retaining the least significant bit of a standard uniform quantizer) as the periodic sketch nonlinearity. Trading for this resource-efficient signature (standard in most acquisition schemes) has almost no impact on the clustering performances, as illustrated by numerical experiments.