MAS-SAM: Segment Any Marine Animal with Aggregated Features

TL;DR

MAS-SAM is a novel marine animal segmentation framework that enhances the Segment Anything Model by integrating specialized adapters, multi-scale feature extraction, and a pyramidal decoder to improve underwater image segmentation accuracy.

Contribution

The paper introduces MAS-SAM, a new framework that adapts SAM for underwater scenes through effective encoders, multi-scale features, and a pyramidal decoder, addressing light scattering and detail loss issues.

Findings

MAS-SAM outperforms existing segmentation methods on four public datasets.

The integration of Hypermap Extraction Module improves multi-scale feature guidance.

The Progressive Prediction Decoder enhances fine-grained segmentation accuracy.

Abstract

Recently, Segment Anything Model (SAM) shows exceptional performance in generating high-quality object masks and achieving zero-shot image segmentation. However, as a versatile vision model, SAM is primarily trained with large-scale natural light images. In underwater scenes, it exhibits substantial performance degradation due to the light scattering and absorption. Meanwhile, the simplicity of the SAM's decoder might lead to the loss of fine-grained object details. To address the above issues, we propose a novel feature learning framework named MAS-SAM for marine animal segmentation, which involves integrating effective adapters into the SAM's encoder and constructing a pyramidal decoder. More specifically, we first build a new SAM's encoder with effective adapters for underwater scenes. Then, we introduce a Hypermap Extraction Module (HEM) to generate multi-scale features for a comprehensive guidance. Finally, we propose a Progressive Prediction Decoder (PPD) to aggregate the multi-scale features and predict the final segmentation results. When grafting with the Fusion Attention Module (FAM), our method enables to extract richer marine information from global contextual cues to fine-grained local details. Extensive experiments on four public MAS datasets demonstrate that our MAS-SAM can obtain better results than other typical segmentation methods. The source code is available at https://github.com/Drchip61/MAS-SAM.