Modal-Adaptive Gated Recoding Network for RGB-D Salient Object Detection

TL;DR

This paper introduces GRNet, a novel RGB-D salient object detection framework that adaptively balances multi-modal information during feature fusion, improving robustness and performance on multiple benchmarks.

Contribution

The paper proposes a modal-adaptive gating mechanism and a recoding mixer to effectively balance and integrate multi-modal features in RGB-D salient object detection.

Findings

Outperforms 9 state-of-the-art methods on eight benchmarks.

Effectively suppresses invalid modal information during feature fusion.

Demonstrates improved robustness and accuracy in real-world scenarios.

Abstract

The multi-modal salient object detection model based on RGB-D information has better robustness in the real world. However, it remains nontrivial to better adaptively balance effective multi-modal information in the feature fusion phase. In this letter, we propose a novel gated recoding network (GRNet) to evaluate the information validity of the two modes, and balance their influence. Our framework is divided into three phases: perception phase, recoding mixing phase and feature integration phase. First, A perception encoder is adopted to extract multi-level single-modal features, which lays the foundation for multi-modal semantic comparative analysis. Then, a modal-adaptive gate unit (MGU) is proposed to suppress the invalid information and transfer the effective modal features to the recoding mixer and the hybrid branch decoder. The recoding mixer is responsible for recoding and mixing the balanced multi-modal information. Finally, the hybrid branch decoder completes the multi-level feature integration under the guidance of an optional edge guidance stream (OEGS). Experiments and analysis on eight popular benchmarks verify that our framework performs favorably against 9 state-of-art methods.