Multi-Modal Image Fusion via Intervention-Stable Feature Learning

TL;DR

This paper introduces an intervention-based causal framework for multi-modal image fusion, focusing on learning robust features that are stable under various perturbations to improve performance and generalization.

Contribution

It proposes a novel causal intervention strategy and a Causal Feature Integrator to identify and prioritize stable, informative features across modalities, enhancing robustness over existing methods.

Findings

Achieves state-of-the-art results on benchmark datasets

Demonstrates robustness under distribution shifts

Improves high-level vision task performance

Abstract

Multi-modal image fusion integrates complementary information from different modalities into a unified representation. Current methods predominantly optimize statistical correlations between modalities, often capturing dataset-induced spurious associations that degrade under distribution shifts. In this paper, we propose an intervention-based framework inspired by causal principles to identify robust cross-modal dependencies. Drawing insights from Pearl's causal hierarchy, we design three principled intervention strategies to probe different aspects of modal relationships: i) complementary masking with spatially disjoint perturbations tests whether modalities can genuinely compensate for each other's missing information, ii) random masking of identical regions identifies feature subsets that remain informative under partial observability, and iii) modality dropout evaluates the irreplaceable contribution of each modality. Based on these interventions, we introduce a Causal Feature Integrator (CFI) that learns to identify and prioritize intervention-stable features maintaining importance across different perturbation patterns through adaptive invariance gating, thereby capturing robust modal dependencies rather than spurious correlations. Extensive experiments demonstrate that our method achieves SOTA performance on both public benchmarks and downstream high-level vision tasks.