TriFusion-SR: Joint Tri-Modal Medical Image Fusion and SR

TL;DR

This paper introduces TriFusionSR, a novel wavelet-guided diffusion framework that jointly performs tri-modal medical image fusion and super-resolution, significantly improving image quality and diagnostic utility.

Contribution

It presents a unified approach combining wavelet decomposition, frequency-aware interaction, and adaptive fusion for enhanced tri-modal medical image super-resolution.

Findings

Achieves 4.8-12.4% PSNR improvement over previous methods.

Reduces RMSE and LPIPS metrics substantially.

Demonstrates state-of-the-art performance across multiple upsampling scales.

Abstract

Multimodal medical image fusion facilitates comprehensive diagnosis by aggregating complementary structural and functional information, but its effectiveness is limited by resolution degradation and modality discrepancies. Existing approaches typically perform image fusion and super-resolution (SR) in separate stages, leading to artifacts and degraded perceptual quality. These limitations are further amplified in tri-modal settings that combine anatomical modalities (e.g., MRI, CT) with functional scans (e.g., PET, SPECT) due to pronounced frequency domain imbalances. We propose TriFusionSR, a wavelet-guided conditional diffusion framework for joint tri-modal fusion and SR. The framework explicitly decomposes multimodal features into frequency bands using the 2D Discrete Wavelet Transform, enabling frequency-aware crossmodal interaction. We further introduce a Rectified Wavelet Features (RWF) strategy for latent coefficient calibration, followed by an Adaptive Spatial-Frequency Fusion (ASFF) module with gated channel-spatial attention to enable structure-driven multimodal refinement. Extensive experiments demonstrate state-of-the-art performance, achieving 4.8-12.4% PSNR improvement and substantial reductions in RMSE and LPIPS across multiple upsampling scales.