KALAHash: Knowledge-Anchored Low-Resource Adaptation for Deep Hashing

TL;DR

KALAHash introduces a knowledge-anchored low-resource adaptation framework for deep hashing, leveraging class-level textual knowledge and efficient fine-tuning to improve retrieval performance with minimal training data.

Contribution

The paper proposes KALAHash, combining CLoRA and KIDDO to enable effective low-resource adaptation in deep hashing using class knowledge and parameter-efficient methods.

Findings

Achieves 4x data efficiency in low-resource scenarios.

Significantly improves retrieval performance with limited training data.

Demonstrates robustness across multiple datasets.

Abstract

Deep hashing has been widely used for large-scale approximate nearest neighbor search due to its storage and search efficiency. However, existing deep hashing methods predominantly rely on abundant training data, leaving the more challenging scenario of low-resource adaptation for deep hashing relatively underexplored. This setting involves adapting pre-trained models to downstream tasks with only an extremely small number of training samples available. Our preliminary benchmarks reveal that current methods suffer significant performance degradation due to the distribution shift caused by limited training samples. To address these challenges, we introduce Class-Calibration LoRA (CLoRA), a novel plug-and-play approach that dynamically constructs low-rank adaptation matrices by leveraging class-level textual knowledge embeddings. CLoRA effectively incorporates prior class knowledge as anchors, enabling parameter-efficient fine-tuning while maintaining the original data distribution. Furthermore, we propose Knowledge-Guided Discrete Optimization (KIDDO), a framework to utilize class knowledge to compensate for the scarcity of visual information and enhance the discriminability of hash codes. Extensive experiments demonstrate that our proposed method, Knowledge- Anchored Low-Resource Adaptation Hashing (KALAHash), significantly boosts retrieval performance and achieves a 4x data efficiency in low-resource scenarios.