High-level Codes and Fine-grained Weights for Online Multi-modal Hashing Retrieval

TL;DR

This paper introduces HCFW, a novel online multi-modal hashing method that improves long-term hash code consistency and multi-modal data fusion using high-level codes and fine-grained weights, demonstrated through extensive experiments.

Contribution

HCFW is the first online multi-modal hashing approach to incorporate category-level high-level codes and instance-level fine-grained weights for improved performance.

Findings

HCFW achieves superior retrieval accuracy on benchmark datasets.

The method maintains hash code consistency over long-term incremental learning.

HCFW effectively fuses multi-modal data at the instance level.

Abstract

In the real world, multi-modal data often appears in a streaming fashion, and there is a growing demand for similarity retrieval from such non-stationary data, especially at a large scale. In response to this need, online multi-modal hashing has gained significant attention. However, existing online multi-modal hashing methods face challenges related to the inconsistency of hash codes during long-term learning and inefficient fusion of different modalities. In this paper, we present a novel approach to supervised online multi-modal hashing, called High-level Codes, Fine-grained Weights (HCFW). To address these problems, HCFW is designed by its non-trivial contributions from two primary dimensions: 1) Online Hashing Perspective. To ensure the long-term consistency of hash codes, especially in incremental learning scenarios, HCFW learns high-level codes derived from category-level semantics. Besides, these codes are adept at handling the category-incremental challenge. 2) Multi-modal Hashing Aspect. HCFW introduces the concept of fine-grained weights designed to facilitate the seamless fusion of complementary multi-modal data, thereby generating multi-modal weights at the instance level and enhancing the overall hashing performance. A comprehensive battery of experiments conducted on two benchmark datasets convincingly underscores the effectiveness and efficiency of HCFW.