Composite Quantization

TL;DR

This paper introduces near-orthogonal composite quantization, a compact coding method for approximate nearest neighbor search that reduces computation cost while maintaining high accuracy, and demonstrates its effectiveness across multiple datasets and applications.

Contribution

It proposes a novel near-orthogonal composite quantization framework with a theoretical foundation and practical efficiency improvements for approximate nearest neighbor search.

Findings

Outperforms existing methods on benchmark datasets

Reduces search complexity from O(D) to O(M)

Enhances applications like mobile search and inner-product similarity

Abstract

This paper studies the compact coding approach to approximate nearest neighbor search. We introduce a composite quantization framework. It uses the composition of several ($M$) elements, each of which is selected from a different dictionary, to accurately approximate a $D$-dimensional vector, thus yielding accurate search, and represents the data vector by a short code composed of the indices of the selected elements in the corresponding dictionaries. Our key contribution lies in introducing a near-orthogonality constraint, which makes the search efficiency is guaranteed as the cost of the distance computation is reduced to $O(M)$ from $O(D)$ through a distance table lookup scheme. The resulting approach is called near-orthogonal composite quantization. We theoretically justify the equivalence between near-orthogonal composite quantization and minimizing an upper bound of a function formed by jointly considering the quantization error and the search cost according to a generalized triangle inequality. We empirically show the efficacy of the proposed approach over several benchmark datasets. In addition, we demonstrate the superior performances in other three applications: combination with inverted multi-index, quantizing the query for mobile search, and inner-product similarity search.