Advancing Marine Research: UWSAM Framework and UIIS10K Dataset for Precise Underwater Instance Segmentation

TL;DR

This paper introduces UWSAM, a novel efficient underwater instance segmentation model that leverages a new large-scale dataset UIIS10K and knowledge distillation techniques to improve accuracy and reduce computational demands in underwater visual tasks.

Contribution

The paper presents a new large-scale dataset UIIS10K and a novel model UWSAM that uses knowledge distillation and an automatic prompt generator for improved underwater instance segmentation.

Findings

UWSAM outperforms state-of-the-art methods on multiple datasets.

The UIIS10K dataset provides extensive annotated underwater images.

Knowledge distillation enhances model efficiency and accuracy.

Abstract

With recent breakthroughs in large-scale modeling, the Segment Anything Model (SAM) has demonstrated significant potential in a variety of visual applications. However, due to the lack of underwater domain expertise, SAM and its variants face performance limitations in end-to-end underwater instance segmentation tasks, while their higher computational requirements further hinder their application in underwater scenarios. To address this challenge, we propose a large-scale underwater instance segmentation dataset, UIIS10K, which includes 10,048 images with pixel-level annotations for 10 categories. Then, we introduce UWSAM, an efficient model designed for automatic and accurate segmentation of underwater instances. UWSAM efficiently distills knowledge from the SAM ViT-Huge image encoder into the smaller ViT-Small image encoder via the Mask GAT-based Underwater Knowledge Distillation (MG-UKD) method for effective visual representation learning. Furthermore, we design an End-to-end Underwater Prompt Generator (EUPG) for UWSAM, which automatically generates underwater prompts instead of explicitly providing foreground points or boxes as prompts, thus enabling the network to locate underwater instances accurately for efficient segmentation. Comprehensive experimental results show that our model is effective, achieving significant performance improvements over state-of-the-art methods on multiple underwater instance datasets. Datasets and codes are available at https://github.com/LiamLian0727/UIIS10K.