Traffic-Domain Video Question Answering with Automatic Captioning
Ehsan Qasemi, Jonathan M. Francis, Alessandro Oltramari

TL;DR
This paper introduces TRIVIA, a weak-supervision method that enhances traffic-related video question answering by integrating traffic scene knowledge, significantly improving model accuracy in urban traffic scenarios.
Contribution
The paper presents a novel weak-supervision approach, TRIVIA, for infusing traffic domain knowledge into large video-language models for improved traffic scene understanding.
Findings
TRIVIA improves accuracy by 6.5 points (19.88%) over baselines.
Empirical validation on SUTD-TrafficQA demonstrates effectiveness.
Method promotes advancements in traffic-related video question answering.
Abstract
Video Question Answering (VidQA) exhibits remarkable potential in facilitating advanced machine reasoning capabilities within the domains of Intelligent Traffic Monitoring and Intelligent Transportation Systems. Nevertheless, the integration of urban traffic scene knowledge into VidQA systems has received limited attention in previous research endeavors. In this work, we present a novel approach termed Traffic-domain Video Question Answering with Automatic Captioning (TRIVIA), which serves as a weak-supervision technique for infusing traffic-domain knowledge into large video-language models. Empirical findings obtained from the SUTD-TrafficQA task highlight the substantial enhancements achieved by TRIVIA, elevating the accuracy of representative video-language models by a remarkable 6.5 points (19.88%) compared to baseline settings. This pioneering methodology holds great promise for…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Human Pose and Action Recognition · Anomaly Detection Techniques and Applications
