Lightweight RGB-D Salient Object Detection from a Speed-Accuracy Tradeoff Perspective

TL;DR

SATNet is a lightweight RGB-D salient object detection network that balances speed and accuracy by enhancing depth quality, modality fusion, and feature representation, outperforming heavyweight models while maintaining high efficiency.

Contribution

The paper introduces SATNet, a novel lightweight RGB-D SOD framework with modules for high-quality depth generation, decoupled modality fusion, and enriched feature representation, achieving state-of-the-art results.

Findings

Outperforms state-of-the-art CNN-based models.

Achieves 415 FPS with only 5.2 million parameters.

Demonstrates superior accuracy on five public datasets.

Abstract

Current RGB-D methods usually leverage large-scale backbones to improve accuracy but sacrifice efficiency. Meanwhile, several existing lightweight methods are difficult to achieve high-precision performance. To balance the efficiency and performance, we propose a Speed-Accuracy Tradeoff Network (SATNet) for Lightweight RGB-D SOD from three fundamental perspectives: depth quality, modality fusion, and feature representation. Concerning depth quality, we introduce the Depth Anything Model to generate high-quality depth maps,which effectively alleviates the multi-modal gaps in the current datasets. For modality fusion, we propose a Decoupled Attention Module (DAM) to explore the consistency within and between modalities. Here, the multi-modal features are decoupled into dual-view feature vectors to project discriminable information of feature maps. For feature representation, we develop a Dual Information Representation Module (DIRM) with a bi-directional inverted framework to enlarge the limited feature space generated by the lightweight backbones. DIRM models texture features and saliency features to enrich feature space, and employ two-way prediction heads to optimal its parameters through a bi-directional backpropagation. Finally, we design a Dual Feature Aggregation Module (DFAM) in the decoder to aggregate texture and saliency features. Extensive experiments on five public RGB-D SOD datasets indicate that the proposed SATNet excels state-of-the-art (SOTA) CNN-based heavyweight models and achieves a lightweight framework with 5.2 M parameters and 415 FPS.