GCAgent: Long-Video Understanding via Schematic and Narrative Episodic Memory

TL;DR

GCAgent introduces a structured episodic memory framework that significantly improves long-video understanding in multimodal large language models by modeling event relations and maintaining global context.

Contribution

The paper presents GCAgent, a novel framework with schematic and narrative episodic memory, enabling deep long-video reasoning and surpassing existing models in accuracy and contextual understanding.

Findings

Achieves up to 23.5% accuracy improvement on Video-MME Long split.

Establishes state-of-the-art performance among 7B-scale MLLMs.

Validates the effectiveness of structured memory for long-video reasoning.

Abstract

Long-video understanding remains a significant challenge for Multimodal Large Language Models (MLLMs) due to inherent token limitations and the complexity of capturing long-term temporal dependencies. Existing methods often fail to capture the global context and complex event relationships necessary for deep video reasoning. To address this, we introduce GCAgent, a novel Global-Context-Aware Agent framework that achieves comprehensive long-video understanding. Our core innovation is the Schematic and Narrative Episodic Memory. This memory structurally models events and their causal and temporal relations into a concise, organized context, fundamentally resolving the long-term dependency problem. Operating in a multi-stage Perception-Action-Reflection cycle, our GCAgent utilizes a Memory Manager to retrieve relevant episodic context for robust, context-aware inference. Extensive experiments confirm that GCAgent significantly enhances long-video understanding, achieving up to 23.5\% accuracy improvement on the Video-MME Long split over a strong MLLM baseline. Furthermore, our framework establishes state-of-the-art performance among comparable 7B-scale MLLMs, achieving 73.4\% accuracy on the Long split and the highest overall average (71.9\%) on the Video-MME benchmark, validating our agent-based reasoning paradigm and structured memory for cognitively-inspired long-video understanding.