Inverted Semantic-Index for Image Retrieval

TL;DR

This paper introduces an inverted semantic-index for large-scale image retrieval that leverages image classification for partitioning, combined with product quantization, to improve retrieval accuracy and candidate list quality.

Contribution

It replaces traditional unsupervised clustering with image classification in index construction and proposes a merging-splitting method to adapt the number of partitions.

Findings

Significantly improved retrieval accuracy on benchmarks.

Enhanced candidate list quality with semantic-aware indexing.

Effective combination of semantic index with product quantization.

Abstract

This paper addresses the construction of inverted index for large-scale image retrieval. The inverted index proposed by J. Sivic brings a significant acceleration by reducing distance computations with only a small fraction of the database. The state-of-the-art inverted indices aim to build finer partitions that produce a concise and accurate candidate list. However, partitioning in these frameworks is generally achieved by unsupervised clustering methods which ignore the semantic information of images. In this paper, we replace the clustering method with image classification, during the construction of codebook. We then propose a merging and splitting method to solve the problem that the number of partitions is unchangeable in the inverted semantic-index. Next, we combine our semantic-index with the product quantization (PQ) so as to alleviate the accuracy loss caused by PQ compression. Finally, we evaluate our model on large-scale image retrieval benchmarks. Experiment results demonstrate that our model can significantly improve the retrieval accuracy by generating high-quality candidate lists.