MambaVF: State Space Model for Efficient Video Fusion

TL;DR

MambaVF introduces a state space model-based framework for efficient video fusion that eliminates explicit motion estimation, significantly reducing computational costs while achieving state-of-the-art results across various video fusion tasks.

Contribution

It reformulates video fusion as a sequential state update process using state space models, enabling long-range temporal modeling with linear complexity and high efficiency.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Reduces parameters by up to 92.25% and FLOPs by 88.79%.

Provides a 2.1x speedup over existing methods.

Abstract

Video fusion is a fundamental technique in various video processing tasks. However, existing video fusion methods heavily rely on optical flow estimation and feature warping, resulting in severe computational overhead and limited scalability. This paper presents MambaVF, an efficient video fusion framework based on state space models (SSMs) that performs temporal modeling without explicit motion estimation. First, by reformulating video fusion as a sequential state update process, MambaVF captures long-range temporal dependencies with linear complexity while significantly reducing computation and memory costs. Second, MambaVF proposes a lightweight SSM-based fusion module that replaces conventional flow-guided alignment via a spatio-temporal bidirectional scanning mechanism. This module enables efficient information aggregation across frames. Extensive experiments across multiple benchmarks demonstrate that our MambaVF achieves state-of-the-art performance in multi-exposure, multi-focus, infrared-visible, and medical video fusion tasks. We highlight that MambaVF enjoys high efficiency, reducing up to 92.25% of parameters and 88.79% of computational FLOPs and a 2.1x speedup compared to existing methods. Project page: https://mambavf.github.io