Multi-Scale and Detail-Enhanced Segment Anything Model for Salient Object Detection

TL;DR

This paper introduces MDSAM, a novel model that enhances the Segment Anything Model for Salient Object Detection by integrating multi-scale, multi-level, and fine-grained detail information, achieving superior performance and generalization.

Contribution

The paper proposes a lightweight multi-scale adapter, a multi-level fusion module, and a detail enhancement module to improve SAM's performance in SOD tasks.

Findings

Outperforms existing SOD methods on multiple datasets.

Demonstrates strong generalization to other segmentation tasks.

Effectively incorporates multi-scale and fine-grained details.

Abstract

Salient Object Detection (SOD) aims to identify and segment the most prominent objects in images. Advanced SOD methods often utilize various Convolutional Neural Networks (CNN) or Transformers for deep feature extraction. However, these methods still deliver low performance and poor generalization in complex cases. Recently, Segment Anything Model (SAM) has been proposed as a visual fundamental model, which gives strong segmentation and generalization capabilities. Nonetheless, SAM requires accurate prompts of target objects, which are unavailable in SOD. Additionally, SAM lacks the utilization of multi-scale and multi-level information, as well as the incorporation of fine-grained details. To address these shortcomings, we propose a Multi-scale and Detail-enhanced SAM (MDSAM) for SOD. Specifically, we first introduce a Lightweight Multi-Scale Adapter (LMSA), which allows SAM to learn multi-scale information with very few trainable parameters. Then, we propose a Multi-Level Fusion Module (MLFM) to comprehensively utilize the multi-level information from the SAM's encoder. Finally, we propose a Detail Enhancement Module (DEM) to incorporate SAM with fine-grained details. Experimental results demonstrate the superior performance of our model on multiple SOD datasets and its strong generalization on other segmentation tasks. The source code is released at https://github.com/BellyBeauty/MDSAM.