FusionFM: All-in-One Multi-Modal Image Fusion with Flow Matching

TL;DR

FusionFM introduces a unified, efficient multi-modal image fusion method using flow matching, pseudo-labels, and continual learning techniques to improve performance, scalability, and sampling speed across various tasks.

Contribution

It proposes a novel flow matching-based probabilistic transport framework for multi-modal image fusion, incorporating pseudo-label selection, a Fusion Refiner, and continual learning strategies.

Findings

Achieves competitive results across diverse fusion tasks.

Significantly improves sampling efficiency.

Maintains a lightweight model design.

Abstract

Current multi-modal image fusion methods typically rely on task-specific models, leading to high training costs and limited scalability. While generative methods provide a unified modeling perspective, they often suffer from slow inference due to the complex sampling trajectories from noise to image. To address this, we formulate image fusion as a direct probabilistic transport from source modalities to the fused image distribution, leveraging the flow matching paradigm to improve sampling efficiency and structural consistency. To mitigate the lack of high-quality fused images for supervision, we collect fusion results from multiple state-of-the-art models as priors, and employ a task-aware selection function to select the most reliable pseudo-labels for each task. We further introduce a Fusion Refiner module that employs a divide-and-conquer strategy to systematically identify, decompose, and enhance degraded components in selected pseudo-labels. For multi-task scenarios, we integrate elastic weight consolidation and experience replay mechanisms to preserve cross-task performance and enhance continual learning ability from both parameter stability and memory retention perspectives. Our approach achieves competitive performance across diverse fusion tasks, while significantly improving sampling efficiency and maintaining a lightweight model design. The code will be available at: https://github.com/Ist-Zhy/FusionFM.