Kernel $k$-Medoids as General Vector Quantization

TL;DR

This paper uncovers a fundamental connection between $k$-medoids clustering and Kernel Density Estimation-based vector quantization through QUBO formulations, revealing their structural similarities and geometric insights.

Contribution

It demonstrates that KDE-based VQ is a special case of $k$-medoids QUBO, providing a unified perspective on these two approaches.

Findings

KDE-QUBO is a special case of $k$-medoids-QUBO under certain conditions.

Reveals structural and geometric relationships between $k$-medoids and KDE-based VQ.

Provides new insights into the weighting parameters in QUBO formulations.

Abstract

Vector Quantization (VQ) is a widely used technique in machine learning and data compression, valued for its simplicity and interpretability. Among hard VQ methods, $k$-medoids clustering and Kernel Density Estimation (KDE) approaches represent two prominent yet seemingly unrelated paradigms -- one distance-based, the other rooted in probability density matching. In this paper, we investigate their connection through the lens of Quadratic Unconstrained Binary Optimization (QUBO). We compare a heuristic QUBO formulation for $k$-medoids, which balances centrality and diversity, with a principled QUBO derived from minimizing Maximum Mean Discrepancy in KDE-based VQ. Surprisingly, we show that the KDE-QUBO is a special case of the $k$-medoids-QUBO under mild assumptions on the kernel's feature map. This reveals a deeper structural relationship between these two approaches and provides new insight into the geometric interpretation of the weighting parameters used in QUBO formulations for VQ.