DIFF-MF: A Difference-Driven Channel-Spatial State Space Model for Multi-Modal Image Fusion

TL;DR

DIFF-MF introduces a difference-driven state space model for multi-modal image fusion, effectively integrating features across channel and spatial dimensions to produce high-quality fused images with improved detail and thermal salience.

Contribution

The paper presents a novel difference-driven channel-spatial state space model that enhances multi-modal image fusion by leveraging feature discrepancy maps and cross-attention modules for adaptive, comprehensive fusion.

Findings

Outperforms existing methods in visual quality and quantitative metrics

Effectively captures global dependencies with linear complexity

Enhances thermal target salience while preserving visible details

Abstract

Multi-modal image fusion aims to integrate complementary information from multiple source images to produce high-quality fused images with enriched content. Although existing approaches based on state space model have achieved satisfied performance with high computational efficiency, they tend to either over-prioritize infrared intensity at the cost of visible details, or conversely, preserve visible structure while diminishing thermal target salience. To overcome these challenges, we propose DIFF-MF, a novel difference-driven channel-spatial state space model for multi-modal image fusion. Our approach leverages feature discrepancy maps between modalities to guide feature extraction, followed by a fusion process across both channel and spatial dimensions. In the channel dimension, a channel-exchange module enhances channel-wise interaction through cross-attention dual state space modeling, enabling adaptive feature reweighting. In the spatial dimension, a spatial-exchange module employs cross-modal state space scanning to achieve comprehensive spatial fusion. By efficiently capturing global dependencies while maintaining linear computational complexity, DIFF-MF effectively integrates complementary multi-modal features. Experimental results on the driving scenarios and low-altitude UAV datasets demonstrate that our method outperforms existing approaches in both visual quality and quantitative evaluation.