\emph{cm}SalGAN: RGB-D Salient Object Detection with Cross-View Generative Adversarial Networks

TL;DR

This paper introduces cmSalGAN, a novel adversarial network that effectively fuses RGB and depth data for salient object detection, achieving state-of-the-art results through cross-view learning and attention mechanisms.

Contribution

The paper proposes a new cross-modality adversarial network with attention and edge modules for improved RGB-D salient object detection.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Effectively models intra- and cross-modality saliency cues.

End-to-end trainable framework with attention and edge detection.

Abstract

Image salient object detection (SOD) is an active research topic in computer vision and multimedia area. Fusing complementary information of RGB and depth has been demonstrated to be effective for image salient object detection which is known as RGB-D salient object detection problem. The main challenge for RGB-D salient object detection is how to exploit the salient cues of both intra-modality (RGB, depth) and cross-modality simultaneously which is known as cross-modality detection problem. In this paper, we tackle this challenge by designing a novel cross-modality Saliency Generative Adversarial Network (\emph{cm}SalGAN). \emph{cm}SalGAN aims to learn an optimal view-invariant and consistent pixel-level representation for RGB and depth images via a novel adversarial learning framework, which thus incorporates both information of intra-view and correlation information of cross-view images simultaneously for RGB-D saliency detection problem. To further improve the detection results, the attention mechanism and edge detection module are also incorporated into \emph{cm}SalGAN. The entire \emph{cm}SalGAN can be trained in an end-to-end manner by using the standard deep neural network framework. Experimental results show that \emph{cm}SalGAN achieves the new state-of-the-art RGB-D saliency detection performance on several benchmark datasets.