RoadscapesQA: A Multitask, Multimodal Dataset for Visual Question Answering on Indian Roads

TL;DR

RoadscapesQA introduces a comprehensive multimodal dataset of Indian road scenes with QA pairs, enabling advancements in autonomous driving scene understanding in diverse, unstructured environments.

Contribution

The paper presents RoadscapesQA, a novel large-scale dataset with images, annotations, and QA pairs for Indian road scenes, supporting research in vision-language models for autonomous driving.

Findings

Dataset includes 9,000 images from diverse Indian environments.

Initial baselines demonstrate the dataset's utility for QA tasks.

The dataset covers urban, rural, daytime, and nighttime scenes.

Abstract

Understanding road scenes is essential for autonomous driving, as it enables systems to interpret visual surroundings to aid in effective decision-making. We present Roadscapes, a multitask multimodal dataset consisting of upto 9,000 images captured in diverse Indian driving environments, accompanied by manually verified bounding boxes. To facilitate scalable scene understanding, we employ rule-based heuristics to infer various scene attributes, which are subsequently used to generate question-answer (QA) pairs for tasks such as object grounding, reasoning, and scene understanding. The dataset includes a variety of scenes from urban and rural India, encompassing highways, service roads, village paths, and congested city streets, captured in both daytime and nighttime settings. Roadscapes has been curated to advance research on visual scene understanding in unstructured environments. In this paper, we describe the data collection and annotation process, present key dataset statistics, and provide initial baselines for image QA tasks using vision-language models.