WIFE-Fusion:Wavelet-aware Intra-inter Frequency Enhancement for Multi-model Image Fusion

TL;DR

WIFE-Fusion introduces a novel frequency-aware multimodal image fusion framework that leverages intra- and inter-frequency interactions to improve feature extraction and fusion quality across diverse datasets.

Contribution

It proposes the first wavelet-aware intra-inter frequency enhancement framework with innovative self-attention and interaction modules for multimodal image fusion.

Findings

Outperforms existing fusion methods on five datasets

Effectively captures cross-modal frequency correlations

Enhances feature extraction and fusion accuracy

Abstract

Multimodal image fusion effectively aggregates information from diverse modalities, with fused images playing a crucial role in vision systems. However, existing methods often neglect frequency-domain feature exploration and interactive relationships. In this paper, we propose wavelet-aware Intra-inter Frequency Enhancement Fusion (WIFE-Fusion), a multimodal image fusion framework based on frequency-domain components interactions. Its core innovations include: Intra-Frequency Self-Attention (IFSA) that leverages inherent cross-modal correlations and complementarity through interactive self-attention mechanisms to extract enriched frequency-domain features, and Inter-Frequency Interaction (IFI) that enhances enriched features and filters latent features via combinatorial interactions between heterogeneous frequency-domain components across modalities. These processes achieve precise source feature extraction and unified modeling of feature extraction-aggregation. Extensive experiments on five datasets across three multimodal fusion tasks demonstrate WIFE-Fusion's superiority over current specialized and unified fusion methods. Our code is available at https://github.com/Lmmh058/WIFE-Fusion.