MaskMed: Decoupled Mask and Class Prediction for Medical Image Segmentation

TL;DR

MaskMed introduces a decoupled segmentation approach with a deformable transformer module, significantly improving medical image segmentation accuracy by enabling flexible feature sharing and full-scale spatial attention.

Contribution

The paper presents a novel decoupled segmentation head and a deformable transformer module, enhancing feature sharing and spatial alignment in medical image segmentation.

Findings

Surpasses nnUNet by +2.0% Dice on AMOS 2022

Achieves +6.9% Dice improvement on BTCV

Demonstrates state-of-the-art segmentation performance

Abstract

Medical image segmentation typically adopts a point-wise convolutional segmentation head to predict dense labels, where each output channel is heuristically tied to a specific class. This rigid design limits both feature sharing and semantic generalization. In this work, we propose a unified decoupled segmentation head that separates multi-class prediction into class-agnostic mask prediction and class label prediction using shared object queries. Furthermore, we introduce a Full-Scale Aware Deformable Transformer module that enables low-resolution encoder features to attend across full-resolution encoder features via deformable attention, achieving memory-efficient and spatially aligned full-scale fusion. Our proposed method, named MaskMed, achieves state-of-the-art performance, surpassing nnUNet by +2.0% Dice on AMOS 2022 and +6.9% Dice on BTCV.