Adaptive Structural Similarity Preserving for Unsupervised Cross Modal Hashing

TL;DR

This paper introduces ASSPH, an unsupervised cross-modal hashing framework that adaptively preserves structural similarities and enhances semantic correlation mining during training, improving multimodal data retrieval.

Contribution

The paper proposes a novel adaptive learning scheme and asymmetric structural semantic representation for unsupervised cross-modal hashing, addressing limitations of static metrics and pairwise association focus.

Findings

Outperforms existing methods in retrieval tasks

Enriches semantic correlations during training

Ensures smooth convergence of the hashing process

Abstract

Cross-modal hashing is an important approach for multimodal data management and application. Existing unsupervised cross-modal hashing algorithms mainly rely on data features in pre-trained models to mine their similarity relationships. However, their optimization objectives are based on the static metric between the original uni-modal features, without further exploring data correlations during the training. In addition, most of them mainly focus on association mining and alignment among pairwise instances in continuous space but ignore the latent structural correlations contained in the semantic hashing space. In this paper, we propose an unsupervised hash learning framework, namely Adaptive Structural Similarity Preservation Hashing (ASSPH), to solve the above problems. Firstly, we propose an adaptive learning scheme, with limited data and training batches, to enrich semantic correlations of unlabeled instances during the training process and meanwhile to ensure a smooth convergence of the training process. Secondly, we present an asymmetric structural semantic representation learning scheme. We introduce structural semantic metrics based on graph adjacency relations during the semantic reconstruction and correlation mining stage and meanwhile align the structure semantics in the hash space with an asymmetric binary optimization process. Finally, we conduct extensive experiments to validate the enhancements of our work in comparison with existing works.