FreqU-FNet: Frequency-Aware U-Net for Imbalanced Medical Image Segmentation

TL;DR

FreqU-FNet introduces a frequency domain U-Net architecture with spectral feature extraction and a frequency-aware loss, significantly improving imbalanced medical image segmentation especially for minority classes.

Contribution

It presents a novel frequency domain segmentation framework with multi-scale spectral features and a frequency-aware loss, addressing class imbalance in medical imaging.

Findings

Outperforms CNN and Transformer baselines on multiple benchmarks.

Effectively captures minority class signals through spectral features.

Enhances segmentation accuracy for under-represented classes.

Abstract

Medical image segmentation faces persistent challenges due to severe class imbalance and the frequency-specific distribution of anatomical structures. Most conventional CNN-based methods operate in the spatial domain and struggle to capture minority class signals, often affected by frequency aliasing and limited spectral selectivity. Transformer-based models, while powerful in modeling global dependencies, tend to overlook critical local details necessary for fine-grained segmentation. To overcome these limitations, we propose FreqU-FNet, a novel U-shaped segmentation architecture operating in the frequency domain. Our framework incorporates a Frequency Encoder that leverages Low-Pass Frequency Convolution and Daubechies wavelet-based downsampling to extract multi-scale spectral features. To reconstruct fine spatial details, we introduce a Spatial Learnable Decoder (SLD) equipped with an adaptive multi-branch upsampling strategy. Furthermore, we design a frequency-aware loss (FAL) function to enhance minority class learning. Extensive experiments on multiple medical segmentation benchmarks demonstrate that FreqU-FNet consistently outperforms both CNN and Transformer baselines, particularly in handling under-represented classes, by effectively exploiting discriminative frequency bands.