Interactive Spatial-Frequency Fusion Mamba for Multi-Modal Image Fusion

TL;DR

This paper introduces a novel interactive fusion framework for multi-modal image fusion that adaptively combines spatial and frequency features across multiple scales, leading to improved fusion performance.

Contribution

The paper proposes an innovative Interactive Spatial-Frequency Fusion Mamba (ISFM) framework that models long-range dependencies, adaptively fuses multi-scale frequency components, and guides spatial features interactively.

Findings

Outperforms state-of-the-art methods on six datasets

Effectively models long-range dependencies with linear complexity

Enhances feature representations through multi-scale frequency integration

Abstract

Multi-Modal Image Fusion (MMIF) aims to combine images from different modalities to produce fused images, retaining texture details and preserving significant information. Recently, some MMIF methods incorporate frequency domain information to enhance spatial features. However, these methods typically rely on simple serial or parallel spatial-frequency fusion without interaction. In this paper, we propose a novel Interactive Spatial-Frequency Fusion Mamba (ISFM) framework for MMIF. Specifically, we begin with a Modality-Specific Extractor (MSE) to extract features from different modalities. It models long-range dependencies across the image with linear computational complexity. To effectively leverage frequency information, we then propose a Multi-scale Frequency Fusion (MFF). It adaptively integrates low-frequency and high-frequency components across multiple scales, enabling robust representations of frequency features. More importantly, we further propose an Interactive Spatial-Frequency Fusion (ISF). It incorporates frequency features to guide spatial features across modalities, enhancing complementary representations. Extensive experiments are conducted on six MMIF datasets. The experimental results demonstrate that our ISFM can achieve better performances than other state-of-the-art methods. The source code is available at https://github.com/Namn23/ISFM.