Online Enhanced Semantic Hashing: Towards Effective and Efficient Retrieval for Streaming Multi-Modal Data

TL;DR

This paper introduces OASIS, an online multi-modal hashing method that efficiently handles streaming data and new unseen classes, improving retrieval performance over existing batch and online models.

Contribution

The paper proposes a novel online hashing model with semantic-enhanced representations and an efficient optimization algorithm for streaming multi-modal data with new classes.

Findings

OASIS outperforms state-of-the-art models in experiments.

The semantic-enhanced representation improves handling of new classes.

The method is efficient and suitable for real-time streaming data.

Abstract

With the vigorous development of multimedia equipment and applications, efficient retrieval of large-scale multi-modal data has become a trendy research topic. Thereinto, hashing has become a prevalent choice due to its retrieval efficiency and low storage cost. Although multi-modal hashing has drawn lots of attention in recent years, there still remain some problems. The first point is that existing methods are mainly designed in batch mode and not able to efficiently handle streaming multi-modal data. The second point is that all existing online multi-modal hashing methods fail to effectively handle unseen new classes which come continuously with streaming data chunks. In this paper, we propose a new model, termed Online enhAnced SemantIc haShing (OASIS). We design novel semantic-enhanced representation for data, which could help handle the new coming classes, and thereby construct the enhanced semantic objective function. An efficient and effective discrete online optimization algorithm is further proposed for OASIS. Extensive experiments show that our method can exceed the state-of-the-art models. For good reproducibility and benefiting the community, our code and data are already available in supplementary material and will be made publicly available.