Generating Roadside LiDAR Datasets from Vehicle-Side Datasets via Novel View Synthesis

TL;DR

This paper presents VRS, a framework that synthesizes labeled roadside LiDAR data from vehicle-side data using novel view synthesis, addressing data scarcity for roadside perception models.

Contribution

VRS introduces a novel view synthesis method with domain gap mitigation techniques to generate scalable, labeled roadside LiDAR datasets from vehicle data.

Findings

Synthesized data improves roadside 3D object detection performance.

VRS effectively compensates for missing geometry and viewpoint changes.

The approach enhances generalization to unseen roadside viewpoints.

Abstract

Intelligent Transportation Systems (ITS) require reliable environmental perception to support safe and efficient transportation. With the rapid development of Vehicle-to-everything (V2X), roadside perception has become an effective means to extend sensing coverage and improve traffic safety. However, the scarcity of large-scale annotated roadside LiDAR datasets poses a major challenge for training high-performance roadside perception models. In this paper, we introduce Vehicle-to-Roadside LiDAR Synthesis (VRS), a data synthesis framework that generates labeled roadside LiDAR datasets from vehicle-side datasets via LiDAR novel view synthesis. To mitigate the vehicle-to-roadside domain gap, VRS employs vehicle point cloud completion to compensate for missing geometry in vehicle-side observations, and introduces an occupancy-based visibility constraint to handle large viewpoint changes during cross-view rendering. The proposed framework enables flexible multi-view rendering for scalable roadside data generation. Extensive experiments on roadside 3D object detection demonstrate that the synthesized data effectively complements real roadside data, mitigates the limitations of limited real-world roadside data, and improves generalization to unseen roadside viewpoints.