MambaVesselNet++: A Hybrid CNN-Mamba Architecture for Medical Image Segmentation

TL;DR

MambaVesselNet++ introduces a hybrid CNN-Mamba architecture that combines local feature extraction with efficient long-range dependency modeling, significantly improving medical image segmentation accuracy across various tasks.

Contribution

The paper presents a novel hybrid CNN-Mamba framework with a texture-aware layer and bifocal fusion decoder, enhancing segmentation performance while reducing computational costs.

Findings

Outperforms existing methods on diverse segmentation tasks

Effectively models long-range dependencies with linear complexity

Achieves superior accuracy in 2D, 3D, and instance segmentation

Abstract

Medical image segmentation plays an important role in computer-aided diagnosis. Traditional convolution-based U-shape segmentation architectures are usually limited by the local receptive field. Existing vision transformers have been widely applied to diverse medical segmentation frameworks due to their superior capabilities of capturing global contexts. Despite the advantage, the real-world application of vision transformers is challenged by their non-linear self-attention mechanism, requiring huge computational costs. To address this issue, the selective state space model (SSM) Mamba has gained recognition for its adeptness in modeling long-range dependencies in sequential data, particularly noted for its efficient memory costs. In this paper, we propose MambaVesselNet++, a Hybrid CNN-Mamba framework for medical image segmentation. Our MambaVesselNet++ is comprised of a hybrid image encoder (Hi-Encoder) and a bifocal fusion decoder (BF-Decoder). In Hi-Encoder, we first devise the texture-aware layer to capture low-level semantic features by leveraging convolutions. Then, we utilize Mamba to effectively model long-range dependencies with linear complexity. The Bi-Decoder adopts skip connections to combine local and global information of the Hi-Encoder for the accurate generation of segmentation masks. Extensive experiments demonstrate that MambaVesselNet++ outperforms current convolution-based, transformer-based, and Mamba-based state-of-the-arts across diverse medical 2D, 3D, and instance segmentation tasks. The code is available at https://github.com/CC0117/MambaVesselNet.