Exploring Deeper! Segment Anything Model with Depth Perception for Camouflaged Object Detection

TL;DR

This paper presents DSAM, a novel RGB-D segmentation model leveraging the Segment Anything Model with depth perception, achieving state-of-the-art results in camouflaged object detection by effectively integrating RGB and depth features.

Contribution

The paper introduces DSAM, the first SAM-based RGB-D model for camouflaged object detection, combining depth cues with RGB features for improved segmentation accuracy.

Findings

Achieves state-of-the-art performance on COD benchmarks.

Utilizes less training resources compared to existing methods.

Effectively integrates depth perception into SAM for improved segmentation.

Abstract

This paper introduces a new Segment Anything Model with Depth Perception (DSAM) for Camouflaged Object Detection (COD). DSAM exploits the zero-shot capability of SAM to realize precise segmentation in the RGB-D domain. It consists of the Prompt-Deeper Module and the Finer Module. The Prompt-Deeper Module utilizes knowledge distillation and the Bias Correction Module to achieve the interaction between RGB features and depth features, especially using depth features to correct erroneous parts in RGB features. Then, the interacted features are combined with the box prompt in SAM to create a prompt with depth perception. The Finer Module explores the possibility of accurately segmenting highly camouflaged targets from a depth perspective. It uncovers depth cues in areas missed by SAM through mask reversion, self-filtering, and self-attention operations, compensating for its defects in the COD domain. DSAM represents the first step towards the SAM-based RGB-D COD model. It maximizes the utilization of depth features while synergizing with RGB features to achieve multimodal complementarity, thereby overcoming the segmentation limitations of SAM and improving its accuracy in COD. Experimental results on COD benchmarks demonstrate that DSAM achieves excellent segmentation performance and reaches the state-of-the-art (SOTA) on COD benchmarks with less consumption of training resources. The code will be available at https://github.com/guobaoxiao/DSAM.