Specificity-preserving RGB-D Saliency Detection

TL;DR

This paper introduces SPNet, a novel RGB-D saliency detection framework that explicitly preserves modality-specific features while effectively integrating shared information, leading to improved detection performance.

Contribution

The study proposes a new framework with modality-specific networks and shared learning, along with novel modules for cross-modal feature fusion and aggregation, enhancing RGB-D saliency detection.

Findings

Outperforms state-of-the-art methods on six RGB-D SOD benchmarks.

Achieves superior results on three camouflaged object detection benchmarks.

Effectively preserves modality-specific features while integrating shared information.

Abstract

Salient object detection (SOD) on RGB and depth images has attracted more and more research interests, due to its effectiveness and the fact that depth cues can now be conveniently captured. Existing RGB-D SOD models usually adopt different fusion strategies to learn a shared representation from the two modalities (\ie, RGB and depth), while few methods explicitly consider how to preserve modality-specific characteristics. In this study, we propose a novel framework, termed SPNet} (Specificity-preserving network), which benefits SOD performance by exploring both the shared information and modality-specific properties (\eg, specificity). Specifically, we propose to adopt two modality-specific networks and a shared learning network to generate individual and shared saliency prediction maps, respectively. To effectively fuse cross-modal features in the shared learning network, we propose a cross-enhanced integration module (CIM) and then propagate the fused feature to the next layer for integrating cross-level information. Moreover, to capture rich complementary multi-modal information for boosting the SOD performance, we propose a multi-modal feature aggregation (MFA) module to integrate the modality-specific features from each individual decoder into the shared decoder. By using a skip connection, the hierarchical features between the encoder and decoder layers can be fully combined. Extensive experiments demonstrate that our~\ours~outperforms cutting-edge approaches on six popular RGB-D SOD and three camouflaged object detection benchmarks. The project is publicly available at: https://github.com/taozh2017/SPNet.