V2U4Real: A Real-world Large-scale Dataset for Vehicle-to-UAV Cooperative Perception

TL;DR

V2U4Real is a large-scale, real-world multi-modal dataset designed for Vehicle-to-UAV cooperative perception, enabling research on overcoming occlusions and enhancing long-range perception in complex environments.

Contribution

This paper introduces V2U4Real, the first extensive real-world dataset for V2U cooperative perception with multi-view sensors, covering diverse scenarios and establishing benchmarks for multiple perception tasks.

Findings

V2U cooperation improves perception robustness.

The dataset enables effective long-range object detection.

State-of-the-art models show significant gains with V2U data.

Abstract

Modern autonomous vehicle perception systems are often constrained by occlusions, blind spots, and limited sensing range. While existing cooperative perception paradigms, such as Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I), have demonstrated their effectiveness in mitigating these challenges, they remain limited to ground-level collaboration and cannot fully address large-scale occlusions or long-range perception in complex environments. To advance research in cross-view cooperative perception, we present V2U4Real, the first large-scale real-world multi-modal dataset for Vehicle-to-UAV (V2U) cooperative object perception. V2U4Real is collected by a ground vehicle and a UAV equipped with multi-view LiDARs and RGB cameras. The dataset covers urban streets, university campuses, and rural roads under diverse traffic scenarios, comprising over 56K LiDAR frames, 56K multi-view camera images, and 700K annotated 3D bounding boxes across four classes. To support a wide range of research tasks, we establish benchmarks for single-agent 3D object detection, cooperative 3D object detection, and object tracking. Comprehensive evaluations of several state-of-the-art models demonstrate the effectiveness of V2U cooperation in enhancing perception robustness and long-range awareness. The V2U4Real dataset and codebase is available at https://github.com/VjiaLi/V2U4Real.