AffectSeek: Agentic Affective Understanding in Long Videos under Vague User Queries

TL;DR

This paper introduces VQAU, a new task and benchmark for affective understanding in long videos driven by vague user queries, and proposes AffectSeek, an agentic framework for multi-step reasoning and evidence grounding.

Contribution

It defines the VQAU task and constructs VQAU-Bench, a comprehensive benchmark, and proposes AffectSeek, an agentic model for effective affective understanding in long videos.

Findings

Existing models struggle with VQAU challenges.

AffectSeek outperforms baseline models on VQAU-Bench.

VQAU remains a difficult task for current affective recognition methods.

Abstract

Existing affective understanding studies have mainly focused on recognizing emotions from images, audio signals, or pre-cliped video clips, where the affective evidence is already given. This passive and clip-centered setting does not fully reflect real-world scenarios, in which users often interact with long videos and express their needs through natural-language queries. In this paper, we study \textbf{Vague-Query-driven video Affective Understanding (VQAU)}, a new task that requires models to localize affective moments in long videos, predict their emotion categories, and generate evidence-grounded rationales under vague user queries. To support this task, we construct \textbf{VQAU-Bench}, a benchmark that integrates long videos, vague affective queries, temporal clip annotations, emotion labels, and rationale explanations into a unified evaluation framework. VQAU-Bench enables systematic assessment of semantic-temporal-affective alignment, affective moment localization, emotion classification, and rationale generation. To address the multi-step reasoning challenges of VQAU, we further propose \textbf{AffectSeek}, an agentic framework that actively seeks, verifies, and explains affective moments in long videos. AffectSeek decomposes VQAU into intent interpretation, candidate localization, clip verification, emotion reasoning, and rationale generation, and progressively aligns vague user intent with long-video evidence through role-specialized reasoning and cross-stage verification. Experiments show that VQAU remains challenging for existing affective recognition models and single-step vision-language models, while AffectSeek provides a simple yet effective framework for agentic long-video affective understanding.