Harmonized Spatial and Spectral Learning for Robust and Generalized Medical Image Segmentation

TL;DR

This paper introduces a novel spectral correlation coefficient objective that, when combined with spatial features, significantly improves the generalization, interpretability, and robustness of medical image segmentation models across various modalities.

Contribution

It proposes a new spectral correlation coefficient objective to enhance domain-generalized segmentation by capturing long-range dependencies and spectral information, improving existing architectures.

Findings

Improved DSC scores in cardiac segmentation (0.81 and 1.63 percentage points)

Enhanced model robustness to noise and better interpretability

Effective across diverse medical imaging modalities

Abstract

Deep learning has demonstrated remarkable achievements in medical image segmentation. However, prevailing deep learning models struggle with poor generalization due to (i) intra-class variations, where the same class appears differently in different samples, and (ii) inter-class independence, resulting in difficulties capturing intricate relationships between distinct objects, leading to higher false negative cases. This paper presents a novel approach that synergies spatial and spectral representations to enhance domain-generalized medical image segmentation. We introduce the innovative Spectral Correlation Coefficient objective to improve the model's capacity to capture middle-order features and contextual long-range dependencies. This objective complements traditional spatial objectives by incorporating valuable spectral information. Extensive experiments reveal that optimizing this objective with existing architectures like UNet and TransUNet significantly enhances generalization, interpretability, and noise robustness, producing more confident predictions. For instance, in cardiac segmentation, we observe a 0.81 pp and 1.63 pp (pp = percentage point) improvement in DSC over UNet and TransUNet, respectively. Our interpretability study demonstrates that, in most tasks, objectives optimized with UNet outperform even TransUNet by introducing global contextual information alongside local details. These findings underscore the versatility and effectiveness of our proposed method across diverse imaging modalities and medical domains.