Supervised Matrix Factorization for Cross-Modality Hashing

TL;DR

This paper introduces SMFH, a novel supervised matrix factorization approach for cross-modality hashing that improves retrieval accuracy by preserving semantic and similarity information across different data modalities.

Contribution

The paper proposes a collective non-matrix factorization algorithm with graph regularization and semantic label integration for enhanced cross-modality hashing.

Findings

SMFH outperforms state-of-the-art methods on three benchmarks.

Incorporating semantic labels improves retrieval accuracy.

Graph regularization effectively preserves multi-modal similarities.

Abstract

Matrix factorization has been recently utilized for the task of multi-modal hashing for cross-modality visual search, where basis functions are learned to map data from different modalities to the same Hamming embedding. In this paper, we propose a novel cross-modality hashing algorithm termed Supervised Matrix Factorization Hashing (SMFH) which tackles the multi-modal hashing problem with a collective non-matrix factorization across the different modalities. In particular, SMFH employs a well-designed binary code learning algorithm to preserve the similarities among multi-modal original features through a graph regularization. At the same time, semantic labels, when available, are incorporated into the learning procedure. We conjecture that all these would facilitate to preserve the most relevant information during the binary quantization process, and hence improve the retrieval accuracy. We demonstrate the superior performance of SMFH on three cross-modality visual search benchmarks, i.e., the PASCAL-Sentence, Wiki, and NUS-WIDE, with quantitative comparison to various state-of-the-art methods