# Weakly-paired Cross-Modal Hashing

## TL;DR

This paper introduces Flex-CMH, a novel cross-modal hashing method that effectively learns from weakly-paired data with incomplete or unknown sample correspondences, improving retrieval performance in practical scenarios.

## Contribution

It proposes a clustering-based matching strategy and a joint optimization framework to handle weakly-paired data in cross-modal hashing, enhancing flexibility and accuracy.

## Key findings

- Flex-CMH outperforms state-of-the-art methods on multi-modal datasets.
- The approach effectively handles incomplete and unknown sample correspondences.
- Experiments demonstrate significant improvements in retrieval accuracy.

## Abstract

Hashing has been widely adopted for large-scale data retrieval in many domains, due to its low storage cost and high retrieval speed. Existing cross-modal hashing methods optimistically assume that the correspondence between training samples across modalities are readily available. This assumption is unrealistic in practical applications. In addition, these methods generally require the same number of samples across different modalities, which restricts their flexibility. We propose a flexible cross-modal hashing approach (Flex-CMH) to learn effective hashing codes from weakly-paired data, whose correspondence across modalities are partially (or even totally) unknown. FlexCMH first introduces a clustering-based matching strategy to explore the local structure of each cluster, and thus to find the potential correspondence between clusters (and samples therein) across modalities. To reduce the impact of an incomplete correspondence, it jointly optimizes in a unified objective function the potential correspondence, the cross-modal hashing functions derived from the correspondence, and a hashing quantitative loss. An alternative optimization technique is also proposed to coordinate the correspondence and hash functions, and to reinforce the reciprocal effects of the two objectives. Experiments on publicly multi-modal datasets show that FlexCMH achieves significantly better results than state-of-the-art methods, and it indeed offers a high degree of flexibility for practical cross-modal hashing tasks.