InterAct-Video: Reasoning-Rich Video QA for Urban Traffic

TL;DR

InterAct-Video introduces a new dataset and benchmark for VideoQA in traffic scenes, highlighting the need for specialized models to understand complex spatiotemporal traffic events.

Contribution

The paper presents the InterAct VideoQA dataset with real-world traffic videos and questions, enabling better evaluation and development of reasoning-rich VideoQA models for urban traffic monitoring.

Findings

State-of-the-art models struggle with complex traffic scenes.

Fine-tuning improves model performance significantly.

The dataset reveals key challenges in traffic scene understanding.

Abstract

Traffic monitoring is crucial for urban mobility, road safety, and intelligent transportation systems (ITS). Deep learning has advanced video-based traffic monitoring through video question answering (VideoQA) models, enabling structured insight extraction from traffic videos. However, existing VideoQA models struggle with the complexity of real-world traffic scenes, where multiple concurrent events unfold across spatiotemporal dimensions. To address these challenges, this paper introduces \textbf{InterAct VideoQA}, a curated dataset designed to benchmark and enhance VideoQA models for traffic monitoring tasks. The InterAct VideoQA dataset comprises 8 hours of real-world traffic footage collected from diverse intersections, segmented into 10-second video clips, with over 25,000 question-answer (QA) pairs covering spatiotemporal dynamics, vehicle interactions, incident detection, and other critical traffic attributes. State-of-the-art VideoQA models are evaluated on InterAct VideoQA, exposing challenges in reasoning over fine-grained spatiotemporal dependencies within complex traffic scenarios. Additionally, fine-tuning these models on InterAct VideoQA yields notable performance improvements, demonstrating the necessity of domain-specific datasets for VideoQA. InterAct VideoQA is publicly available as a benchmark dataset to facilitate future research in real-world deployable VideoQA models for intelligent transportation systems. GitHub Repo: https://github.com/joe-rabbit/InterAct_VideoQA