Traffic-Domain Video Question Answering with Automatic Captioning

TL;DR

This paper introduces TRIVIA, a weak-supervision method that enhances traffic-related video question answering by integrating traffic scene knowledge, significantly improving model accuracy in urban traffic scenarios.

Contribution

The paper presents a novel weak-supervision approach, TRIVIA, for infusing traffic domain knowledge into large video-language models for improved traffic scene understanding.

Findings

TRIVIA improves accuracy by 6.5 points (19.88%) over baselines.

Empirical validation on SUTD-TrafficQA demonstrates effectiveness.

Method promotes advancements in traffic-related video question answering.

Abstract

Video Question Answering (VidQA) exhibits remarkable potential in facilitating advanced machine reasoning capabilities within the domains of Intelligent Traffic Monitoring and Intelligent Transportation Systems. Nevertheless, the integration of urban traffic scene knowledge into VidQA systems has received limited attention in previous research endeavors. In this work, we present a novel approach termed Traffic-domain Video Question Answering with Automatic Captioning (TRIVIA), which serves as a weak-supervision technique for infusing traffic-domain knowledge into large video-language models. Empirical findings obtained from the SUTD-TrafficQA task highlight the substantial enhancements achieved by TRIVIA, elevating the accuracy of representative video-language models by a remarkable 6.5 points (19.88%) compared to baseline settings. This pioneering methodology holds great promise for driving advancements in the field, inspiring researchers and practitioners alike to unlock the full potential of emerging video-language models in traffic-related applications.