MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction

TL;DR

MAVFusion is an efficient end-to-end video fusion framework that uses motion-aware sparse interactions to combine infrared and visible videos, improving speed and quality by focusing on dynamic regions.

Contribution

It introduces a novel motion-aware sparse interaction mechanism that adaptively allocates attention to dynamic regions, significantly enhancing efficiency and fusion quality.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Runs at 14.16 FPS at 640x480 resolution.

Effectively preserves temporal consistency and details.

Abstract

Infrared and visible video fusion combines the object saliency from infrared images with the texture details from visible images to produce semantically rich fusion results. However, most existing methods are designed for static image fusion and cannot effectively handle frame-to-frame motion in videos. Current video fusion methods improve temporal consistency by introducing interactions across frames, but they often require high computational cost. To mitigate these challenges, we propose MAVFusion, an end-to-end video fusion framework featuring a motion-aware sparse interaction mechanism that enhances efficiency while maintaining superior fusion quality. Specifically, we leverage optical flow to identify dynamic regions in multi-modal sequences, adaptively allocating computationally intensive cross-modal attention to these sparse areas to capture salient transitions and facilitate inter-modal information exchange. For static background regions, a lightweight weak interaction module is employed to maintain structural and appearance integrity. By decoupling the processing of dynamic and static regions, MAVFusion simultaneously preserves temporal consistency and fine-grained details while significantly accelerating inference. Extensive experiments demonstrate that MAVFusion achieves state-of-the-art performance on multiple infrared and visible video benchmarks, achieving a speed of 14.16\,FPS at $640 \times 480$ resolution. The source code will be available at https://github.com/ixilai/MAVFusion.