An Attention-based Multi-Scale Feature Learning Network for Multimodal Medical Image Fusion

TL;DR

This paper introduces a novel attention-based multi-scale feature learning network for multimodal medical image fusion, improving the quality and efficiency of fused images for clinical diagnosis.

Contribution

The paper proposes a Dilated Residual Attention Network with a new Softmax-based fusion strategy, advancing the state-of-the-art in multimodal medical image fusion.

Findings

Outperforms existing methods on four fusion metrics

Effectively extracts multi-scale deep semantic features

Enhances diagnostic efficiency with high-quality fused images

Abstract

Medical images play an important role in clinical applications. Multimodal medical images could provide rich information about patients for physicians to diagnose. The image fusion technique is able to synthesize complementary information from multimodal images into a single image. This technique will prevent radiologists switch back and forth between different images and save lots of time in the diagnostic process. In this paper, we introduce a novel Dilated Residual Attention Network for the medical image fusion task. Our network is capable to extract multi-scale deep semantic features. Furthermore, we propose a novel fixed fusion strategy termed Softmax-based weighted strategy based on the Softmax weights and matrix nuclear norm. Extensive experiments show our proposed network and fusion strategy exceed the state-of-the-art performance compared with reference image fusion methods on four commonly used fusion metrics.