VideoMamba: Spatio-Temporal Selective State Space Model
Jinyoung Park, Hee-Seon Kim, Kangwook Ko, Minbeom Kim, Changick Kim

TL;DR
VideoMamba introduces a resource-efficient spatio-temporal state space model for video recognition, capturing complex dependencies with linear complexity, outperforming transformer-based methods in efficiency and effectiveness.
Contribution
It presents a novel spatio-temporal SSM architecture for video recognition that is more efficient than transformers and effectively models long-range dependencies.
Findings
Achieves competitive performance on video benchmarks.
Demonstrates superior efficiency compared to transformer models.
Effectively captures long-range spatio-temporal dependencies.
Abstract
We introduce VideoMamba, a novel adaptation of the pure Mamba architecture, specifically designed for video recognition. Unlike transformers that rely on self-attention mechanisms leading to high computational costs by quadratic complexity, VideoMamba leverages Mamba's linear complexity and selective SSM mechanism for more efficient processing. The proposed Spatio-Temporal Forward and Backward SSM allows the model to effectively capture the complex relationship between non-sequential spatial and sequential temporal information in video. Consequently, VideoMamba is not only resource-efficient but also effective in capturing long-range dependency in videos, demonstrated by competitive performance and outstanding efficiency on a variety of video understanding benchmarks. Our work highlights the potential of VideoMamba as a powerful tool for video understanding, offering a simple yet…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsOpinion Dynamics and Social Influence
