LeAdQA: LLM-Driven Context-Aware Temporal Grounding for Video Question Answering

TL;DR

LeAdQA introduces a novel framework combining causal-aware query refinement and fine-grained visual grounding, significantly improving complex reasoning in VideoQA tasks by leveraging LLMs and adaptive fusion for precise segment retrieval.

Contribution

This paper presents LeAdQA, a new approach that enhances VideoQA by integrating causal-aware query reformulation with targeted visual grounding, addressing limitations of previous methods.

Findings

Achieves state-of-the-art performance on NExT-QA, IntentQA, and NExT-GQA datasets.

Demonstrates improved understanding of causal-temporal structures in videos.

Enhances reasoning accuracy while maintaining computational efficiency.

Abstract

Video Question Answering (VideoQA) requires identifying sparse critical moments in long videos and reasoning about their causal relationships to answer semantically complex questions. While recent advances in multimodal learning have improved alignment and fusion, current approaches remain limited by two prevalent but fundamentally flawed strategies: (1) task-agnostic sampling indiscriminately processes all frames, overwhelming key events with irrelevant content; and (2) heuristic retrieval captures superficial patterns but misses causal-temporal structures needed for complex reasoning. To address these challenges, we introduce LeAdQA, an innovative approach that bridges these gaps through synergizing causal-aware query refinement with fine-grained visual grounding. Our method first leverages LLMs to reformulate question-option pairs, resolving causal ambiguities and sharpening temporal focus. These refined queries subsequently direct a temporal grounding model to precisely retrieve the most salient segments, complemented by an adaptive fusion mechanism dynamically integrating the evidence to maximize relevance. The integrated visual-textual cues are then processed by an MLLM to generate accurate, contextually-grounded answers. Experiments on NExT-QA, IntentQA, and NExT-GQA demonstrate that our method's precise visual grounding substantially enhances the understanding of video-question relationships, achieving state-of-the-art (SOTA) performance on complex reasoning tasks while maintaining computational efficiency.