MamFusion: Multi-Mamba with Temporal Fusion for Partially Relevant Video Retrieval
Xinru Ying, Jiaqi Mo, Jingyang Lin, Canghong Jin, Fangfang Wang, Lina Wei

TL;DR
MamFusion introduces a novel multi-Mamba with temporal fusion framework that significantly improves partially relevant video retrieval by effectively modeling long-term temporal relationships between text queries and video content.
Contribution
The paper proposes MamFusion, a new framework combining multi-Mamba modules with temporal fusion to enhance long-sequence video understanding for PRVR tasks.
Findings
Achieves state-of-the-art retrieval performance on large-scale datasets.
Effectively models temporal relationships between text and video.
Improves contextual understanding in long untrimmed videos.
Abstract
Partially Relevant Video Retrieval (PRVR) is a challenging task in the domain of multimedia retrieval. It is designed to identify and retrieve untrimmed videos that are partially relevant to the provided query. In this work, we investigate long-sequence video content understanding to address information redundancy issues. Leveraging the outstanding long-term state space modeling capability and linear scalability of the Mamba module, we introduce a multi-Mamba module with temporal fusion framework (MamFusion) tailored for PRVR task. This framework effectively captures the state-relatedness in long-term video content and seamlessly integrates it into text-video relevance understanding, thereby enhancing the retrieval process. Specifically, we introduce Temporal T-to-V Fusion and Temporal V-to-T Fusion to explicitly model temporal relationships between text queries and video moments,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques · Video Analysis and Summarization
