LooC: Effective Low-Dimensional Codebook for Compositional Vector Quantization

TL;DR

LooC introduces a low-dimensional, compositional codebook for vector quantization that enhances performance and compactness, enabling better feature approximation and serving as a versatile module for various tasks.

Contribution

The paper presents LooC, a novel low-dimensional, compositional codebook for vector quantization that improves efficiency and performance over existing methods.

Findings

LooC achieves state-of-the-art results on multiple benchmarks.

The method significantly reduces codebook size while maintaining high accuracy.

LooC effectively utilizes full codebook capacity without collapse.

Abstract

Vector quantization (VQ) is a prevalent and fundamental technique that discretizes continuous feature vectors by approximating them using a codebook. As the diversity and complexity of data and models continue to increase, there is an urgent need for high-capacity, yet more compact VQ methods. This paper aims to reconcile this conflict by presenting a new approach called LooC, which utilizes an effective Low-dimensional codebook for Compositional vector quantization. Firstly, LooC introduces a parameter-efficient codebook by reframing the relationship between codevectors and feature vectors, significantly expanding its solution space. Instead of individually matching codevectors with feature vectors, LooC treats them as lower-dimensional compositional units within feature vectors and combines them, resulting in a more compact codebook with improved performance. Secondly, LooC incorporates a parameter-free extrapolation-by-interpolation mechanism to enhance and smooth features during the VQ process, which allows for better preservation of details and fidelity in feature approximation. The design of LooC leads to full codebook usage, effectively utilizing the compact codebook while avoiding the problem of collapse. Thirdly, LooC can serve as a plug-and-play module for existing methods for different downstream tasks based on VQ. Finally, extensive evaluations on different tasks, datasets, and architectures demonstrate that LooC outperforms existing VQ methods, achieving state-of-the-art performance with a significantly smaller codebook.