OVPD: A Virtual-Physical Fusion Testing Dataset of OnSite Auton-omous Driving Challenge

TL;DR

OVPD is a comprehensive, real-vehicle-in-the-loop dataset combining virtual and physical data to enhance autonomous driving testing and validation.

Contribution

It introduces a novel virtual-physical fusion dataset from the 2025 OnSite Autonomous Driving Challenge, enabling realistic closed-loop testing environments.

Findings

Contains nearly 3 hours of multi-modal data from 20 teams.

Supports safety, efficiency, and rule compliance assessment.

Facilitates failure diagnosis and iterative improvement.

Abstract

The rapid iteration of autonomous driving algorithms has created a growing demand for high-fidelity, replayable, and diagnosable testing data. However, many public datasets lack real vehicle dynamics feedback and closed-loop interaction with surrounding traffic and road infrastructure, limiting their ability to reflect deployment readiness. To address this gap, we present OVPD (OnSite Virtual-Physical Dataset), a virtual-physical fusion testing dataset released from the 2025 OnSite Autonomous Driving Challenge. Centered on real-vehicle-in-the-loop testing, OVPD integrates virtual background traffic with vehicle-infrastructure perception to build controllable and interactive closed-loop test environments on a proving ground. The dataset contains 20 testing clips from 20 teams over a scenario chain of 15 atomic scenarios, totaling nearly 3 hours of multi-modal data, including vehicle trajectories and states, control commands, and digital-twin-rendered surround-view observations. OVPD supports long-tail planning and decision-making validation, open-loop or platform-enabled closed-loop evaluation, and comprehensive assessment across safety, efficiency, comfort, rule compliance, and traffic impact, providing actionable evidence for failure diagnosis and iterative improvement. The dataset is available via: https://huggingface.co/datasets/Yuhang253820/Onsite_OPVD