Focal Modulation and Bidirectional Feature Fusion Network for Medical Image Segmentation

TL;DR

This paper introduces FM-BFF-Net, a hybrid CNN-transformer model with focal modulation and bidirectional feature fusion, achieving superior accuracy in diverse medical image segmentation tasks.

Contribution

It proposes a novel network architecture combining focal modulation and bidirectional feature fusion to improve boundary accuracy and robustness in medical image segmentation.

Findings

Outperforms state-of-the-art methods in Jaccard index and Dice coefficient across multiple datasets.

Enhances boundary precision and handles variations in lesion size, shape, and contrast.

Demonstrates effectiveness on polyp detection, skin lesion segmentation, and ultrasound imaging.

Abstract

Medical image segmentation is essential for clinical applications such as disease diagnosis, treatment planning, and disease development monitoring because it provides precise morphological and spatial information on anatomical structures that directly influence treatment decisions. Convolutional neural networks significantly impact image segmentation; however, since convolution operations are local, capturing global contextual information and long-range dependencies is still challenging. Their capacity to precisely segment structures with complicated borders and a variety of sizes is impacted by this restriction. Since transformers use self-attention methods to capture global context and long-range dependencies efficiently, integrating transformer-based architecture with CNNs is a feasible approach to overcoming these challenges. To address these challenges, we propose the Focal Modulation and Bidirectional Feature Fusion Network for Medical Image Segmentation, referred to as FM-BFF-Net in the remainder of this paper. The network combines convolutional and transformer components, employs a focal modulation attention mechanism to refine context awareness, and introduces a bidirectional feature fusion module that enables efficient interaction between encoder and decoder representations across scales. Through this design, FM-BFF-Net enhances boundary precision and robustness to variations in lesion size, shape, and contrast. Extensive experiments on eight publicly available datasets, including polyp detection, skin lesion segmentation, and ultrasound imaging, show that FM-BFF-Net consistently surpasses recent state-of-the-art methods in Jaccard index and Dice coefficient, confirming its effectiveness and adaptability for diverse medical imaging scenarios.