Deep Learning Techniques for Future Intelligent Cross-Media Retrieval

TL;DR

This paper provides a comprehensive survey of deep learning techniques applied to cross-media retrieval, categorizing methods, challenges, datasets, and solutions to advance understanding in this emerging field.

Contribution

It introduces the first extensive review of deep learning approaches for cross-media retrieval, including a taxonomy of challenges and a categorization of methods.

Findings

Deep neural networks effectively bridge the media gap in retrieval tasks.

Unsupervised and supervised deep learning methods are both prominent in current research.

The paper highlights key datasets and challenges for future research in cross-media retrieval.

Abstract

With the advancement in technology and the expansion of broadcasting, cross-media retrieval has gained much attention. It plays a significant role in big data applications and consists in searching and finding data from different types of media. In this paper, we provide a novel taxonomy according to the challenges faced by multi-modal deep learning approaches in solving cross-media retrieval, namely: representation, alignment, and translation. These challenges are evaluated on deep learning (DL) based methods, which are categorized into four main groups: 1) unsupervised methods, 2) supervised methods, 3) pairwise based methods, and 4) rank based methods. Then, we present some well-known cross-media datasets used for retrieval, considering the importance of these datasets in the context in of deep learning based cross-media retrieval approaches. Moreover, we also present an extensive review of the state-of-the-art problems and its corresponding solutions for encouraging deep learning in cross-media retrieval. The fundamental objective of this work is to exploit Deep Neural Networks (DNNs) for bridging the "media gap", and provide researchers and developers with a better understanding of the underlying problems and the potential solutions of deep learning assisted cross-media retrieval. To the best of our knowledge, this is the first comprehensive survey to address cross-media retrieval under deep learning methods.