V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perception

TL;DR

V2X-Radar introduces the first large-scale, real-world multi-modal dataset incorporating 4D Radar for cooperative perception, addressing the gap in datasets that include this sensor type for autonomous vehicle safety in diverse weather and lighting conditions.

Contribution

This paper presents V2X-Radar, a comprehensive multi-modal dataset with 4D Radar, enabling advanced research in cooperative perception for autonomous vehicles, which was previously limited by lack of such datasets.

Findings

Dataset includes 20K LiDAR frames, 40K camera images, 20K 4D Radar data.

Contains 350K annotated boxes across five categories.

Provides benchmarks for cooperative, roadside, and single-vehicle perception.

Abstract

Modern autonomous vehicle perception systems often struggle with occlusions and limited perception range. Previous studies have demonstrated the effectiveness of cooperative perception in extending the perception range and overcoming occlusions, thereby enhancing the safety of autonomous driving. In recent years, a series of cooperative perception datasets have emerged; however, these datasets primarily focus on cameras and LiDAR, neglecting 4D Radar, a sensor used in single-vehicle autonomous driving to provide robust perception in adverse weather conditions. In this paper, to bridge the gap created by the absence of 4D Radar datasets in cooperative perception, we present V2X-Radar, the first large-scale, real-world multi-modal dataset featuring 4D Radar. V2X-Radar dataset is collected using a connected vehicle platform and an intelligent roadside unit equipped with 4D Radar, LiDAR, and multi-view cameras. The collected data encompasses sunny and rainy weather conditions, spanning daytime, dusk, and nighttime, as well as various typical challenging scenarios. The dataset consists of 20K LiDAR frames, 40K camera images, and 20K 4D Radar data, including 350K annotated boxes across five categories. To support various research domains, we have established V2X-Radar-C for cooperative perception, V2X-Radar-I for roadside perception, and V2X-Radar-V for single-vehicle perception. Furthermore, we provide comprehensive benchmarks across these three sub-datasets. We will release all datasets and benchmark codebase at https://huggingface.co/datasets/yanglei18/V2X-Radar and https://github.com/yanglei18/V2X-Radar.