V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception

TL;DR

V2X-Real is a comprehensive large-scale dataset designed to advance Vehicle-to-Everything cooperative perception, featuring multi-modal sensor data from vehicles and infrastructure in urban environments, enabling new research and benchmarking.

Contribution

The paper introduces V2X-Real, the first large-scale real-world dataset supporting diverse V2X cooperation modes with multi-modal data and extensive annotations for perception research.

Findings

Provides 33K LiDAR frames and 171K camera images with 1.2M annotations.

Includes four types of datasets for different cooperative perception modes.

Offers benchmarks for state-of-the-art V2X perception methods.

Abstract

Recent advancements in Vehicle-to-Everything (V2X) technologies have enabled autonomous vehicles to share sensing information to see through occlusions, greatly boosting the perception capability. However, there are no real-world datasets to facilitate the real V2X cooperative perception research -- existing datasets either only support Vehicle-to-Infrastructure cooperation or Vehicle-to-Vehicle cooperation. In this paper, we present V2X-Real, a large-scale dataset that includes a mixture of multiple vehicles and smart infrastructure to facilitate the V2X cooperative perception development with multi-modality sensing data. Our V2X-Real is collected using two connected automated vehicles and two smart infrastructure, which are all equipped with multi-modal sensors including LiDAR sensors and multi-view cameras. The whole dataset contains 33K LiDAR frames and 171K camera data with over 1.2M annotated bounding boxes of 10 categories in very challenging urban scenarios. According to the collaboration mode and ego perspective, we derive four types of datasets for Vehicle-Centric, Infrastructure-Centric, Vehicle-to-Vehicle, and Infrastructure-to-Infrastructure cooperative perception. Comprehensive multi-class multi-agent benchmarks of SOTA cooperative perception methods are provided. The V2X-Real dataset and codebase are available at https://mobility-lab.seas.ucla.edu/v2x-real.