SSFMamba: Learning Symmetry-driven Spatial-Frequency Modeling for Physically Consistent 3D Medical Image Segmentation

TL;DR

SSFMamba introduces a novel symmetry-driven spatial-frequency framework for 3D medical image segmentation, effectively capturing local details and global context while respecting spectral properties, leading to superior performance across MRI and CT datasets.

Contribution

The paper proposes SSFMamba, a dual-branch 3D segmentation model utilizing a multi-directional frequency scanning mechanism that integrates Hermitian symmetry with state space models for enhanced structural modeling.

Findings

Outperforms state-of-the-art methods on BraTS2020, BraTS2023, and BTCV datasets.

Achieves 81.97% Dice score on pancreas segmentation.

Demonstrates robustness across MRI and CT modalities.

Abstract

Accurate 3D medical image segmentation requires a delicate balance between fine-grained local details and global contextual understanding. While spatial-domain models often struggle with long-range dependencies, existing frequency-based approaches frequently overlook intrinsic spectral properties such as Hermitian symmetry, leading to suboptimal feature integration. In this paper, we propose SSFMamba, a Mamba based Symmetry-driven Spatial-Frequency fusion framework tailored for 3D medical imaging. Our architecture employs a complementary dual-branch design: the spatial branch preserves intricate anatomical textures, while the frequency branch captures global contextual dependencies in the frequency domain. A core innovation is the 3D Multi-Directional Scanning Mechanism (MDSM), which integrates Hermitian symmetry with the causal nature of State Space Models (SSMs) to enable direction-aware global modeling. Crucially, by shifting the modeling focus to frequency-domain spectral components, SSFMamba captures the underlying structural characteristics of anatomical tissues. This leads to a highly adaptable framework that excels in both MRI and CT applications, regardless of the significant variations in intensity distributions. Extensive evaluations on the BraTS2020, BraTS2023, and BTCV datasets demonstrate that SSFMamba consistently outperforms state-of-the-art methods. Notably, our approach achieves exceptional performance on low-contrast organs such as the pancreas (81.97% Dice), underscoring its potential as a unified and physically consistent perception framework for diverse 3D clinical applications.