ViT2Hash: Unsupervised Information-Preserving Hashing

TL;DR

This paper introduces ViT2Hash, an unsupervised image hashing method that fine-tunes a pre-trained ViT model with information-preserving modules to generate high-quality binary codes, significantly improving retrieval performance.

Contribution

It proposes a novel unsupervised hashing framework that leverages pre-trained ViT with feature-preserving and hashing-preserving modules for better information retention.

Findings

Achieves higher MAP scores on three benchmark datasets.

Effectively preserves meaningful information during hashing.

Outperforms existing unsupervised hashing methods.

Abstract

Unsupervised image hashing, which maps images into binary codes without supervision, is a compressor with a high compression rate. Hence, how to preserving meaningful information of the original data is a critical problem. Inspired by the large-scale vision pre-training model, known as ViT, which has shown significant progress for learning visual representations, in this paper, we propose a simple information-preserving compressor to finetune the ViT model for the target unsupervised hashing task. Specifically, from pixels to continuous features, we first propose a feature-preserving module, using the corrupted image as input to reconstruct the original feature from the pre-trained ViT model and the complete image, so that the feature extractor can focus on preserving the meaningful information of original data. Secondly, from continuous features to hash codes, we propose a hashing-preserving module, which aims to keep the semantic information from the pre-trained ViT model by using the proposed Kullback-Leibler divergence loss. Besides, the quantization loss and the similarity loss are added to minimize the quantization error. Our method is very simple and achieves a significantly higher degree of MAP on three benchmark image datasets.