UniHash: Unifying Pointwise and Pairwise Hashing Paradigms

TL;DR

UniHash is a dual-branch deep hashing framework that unifies pointwise and pairwise paradigms to improve image retrieval performance across both seen and unseen categories, leveraging mutual learning and a novel exchange module.

Contribution

The paper introduces UniHash, a novel dual-branch framework that combines pointwise and pairwise hashing paradigms with bidirectional knowledge transfer for balanced retrieval.

Findings

Achieves state-of-the-art results on CIFAR-10, MSCOCO, and ImageNet.

Balances retrieval performance on seen and unseen categories.

Demonstrates theoretical and empirical effectiveness of the unified approach.

Abstract

Effective retrieval across both seen and unseen categories is crucial for modern image retrieval systems. Retrieval on seen categories ensures precise recognition of known classes, while retrieval on unseen categories promotes generalization to novel classes with limited supervision. However, most existing deep hashing methods are confined to a single training paradigm, either pointwise or pairwise, where the former excels on seen categories and the latter generalizes better to unseen ones. To overcome this limitation, we propose Unified Hashing (UniHash), a dual-branch framework that unifies the strengths of both paradigms to achieve balanced retrieval performance across seen and unseen categories. UniHash consists of two complementary branches: a center-based branch following the pointwise paradigm and a pairwise branch following the pairwise paradigm. A novel hash code learning method is introduced to enable bidirectional knowledge transfer between branches, improving hash code discriminability and generalization. It employs a mutual learning loss to align hash representations and introduces a Split-Merge Mixture of Hash Experts (SM-MoH) module to enhance cross-branch exchange of hash representations. Theoretical analysis substantiates the effectiveness of UniHash, and extensive experiments on CIFAR-10, MSCOCO, and ImageNet demonstrate that UniHash consistently achieves state-of-the-art performance in both seen and unseen image retrieval scenarios.