# AniDriveQA: a VQA dataset for driving scenes with animal presence

## TL;DR

AniDriveQA is a new dataset designed to test how well vision-language models handle rare driving scenarios involving animals.

## Contribution

The paper introduces AniDriveQA, a novel VQA dataset focused on animal-related driving scenes to evaluate vision-language models.

## Key findings

- AniDriveQA reveals significant performance gaps in VLMs across different driving tasks involving animals.
- The dataset highlights limitations in current models for handling rare and safety-critical autonomous driving scenarios.
- Experiments show large disparities in model performance when evaluating structured and open-ended tasks.

## Abstract

Animal-involved scenarios pose significant challenges for autonomous driving systems due to their rarity, unpredictability, and safety-critical nature. Despite their importance, existing vision-language datasets for autonomous driving largely overlook these long-tail situations.

To address this gap, we introduce AniDriveQA, a novel visual question answering (VQA) dataset specifically designed to evaluate vision-language models (VLMs) in driving scenarios involving animals. The dataset is constructed through a scalable pipeline that collects diverse animal-related traffic scenes from internet videos, filters and annotates them using object detection and scene classification models, and generates multi-task VQA labels with a large vision-language model. AniDriveQA includes three key task types: scene description, animal description, and driving suggestion.

For evaluation, a hybrid scheme was employed that combined classification accuracy for structured tasks with LLM-based scoring for open-ended responses. Extensive experiments on various open-source VLMs revealed large performance disparities across models and task types.

The experimental results demonstrate that AniDriveQA effectively exposes the limitations of current VLMs in rare yet safety-critical autonomous driving scenarios. The dataset provides a valuable diagnostic benchmark for advancing reasoning, perception, and decision-making capabilities in future vision-language models.