SpectralMamba-UNet: Frequency-Disentangled State Space Modeling for Texture-Structure Consistent Medical Image Segmentation

TL;DR

SpectralMamba-UNet introduces a frequency-disentangled spectral domain approach for medical image segmentation, effectively modeling both global structures and fine details, leading to improved accuracy across multiple benchmarks.

Contribution

It proposes a novel spectral domain framework with spectral decomposition, reweighting, and fusion modules to enhance boundary detail preservation and global context modeling.

Findings

Consistent performance improvements on five public benchmarks.

Effective separation of structural and textural features in spectral domain.

Enhanced boundary and global structure modeling in medical images.

Abstract

Accurate medical image segmentation requires effective modeling of both global anatomical structures and fine-grained boundary details. Recent state space models (e.g., Vision Mamba) offer efficient long-range dependency modeling. However, their one-dimensional serialization weakens local spatial continuity and high-frequency representation. To this end, we propose SpectralMamba-UNet, a novel frequency-disentangled framework to decouple the learning of structural and textural information in the spectral domain. Our Spectral Decomposition and Modeling (SDM) module applies discrete cosine transform to decompose low- and high-frequency features, where low frequency contributes to global contextual modeling via a frequency-domain Mamba and high frequency preserves boundary-sensitive details. To balance spectral contributions, we introduce a Spectral Channel Reweighting (SCR) mechanism to form channel-wise frequency-aware attention, and a Spectral-Guided Fusion (SGF) module to achieve adaptively multi-scale fusion in the decoder. Experiments on five public benchmarks demonstrate consistent improvements across diverse modalities and segmentation targets, validating the effectiveness and generalizability of our approach.