CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splatting

TL;DR

CRUISE is a novel framework that reconstructs and edits real-world V2X driving scenes using Gaussian Splatting, enhancing dataset augmentation and improving 3D detection and tracking performance.

Contribution

CRUISE introduces a decomposed Gaussian Splatting method for accurate scene reconstruction and flexible editing in V2X environments, enabling large-scale data augmentation.

Findings

High-fidelity scene reconstruction demonstrated

Improved 3D detection and tracking performance

Effective generation of challenging corner cases

Abstract

Vehicle-to-everything (V2X) communication plays a crucial role in autonomous driving, enabling cooperation between vehicles and infrastructure. While simulation has significantly contributed to various autonomous driving tasks, its potential for data generation and augmentation in V2X scenarios remains underexplored. In this paper, we introduce CRUISE, a comprehensive reconstruction-and-synthesis framework designed for V2X driving environments. CRUISE employs decomposed Gaussian Splatting to accurately reconstruct real-world scenes while supporting flexible editing. By decomposing dynamic traffic participants into editable Gaussian representations, CRUISE allows for seamless modification and augmentation of driving scenes. Furthermore, the framework renders images from both ego-vehicle and infrastructure views, enabling large-scale V2X dataset augmentation for training and evaluation. Our experimental results demonstrate that: 1) CRUISE reconstructs real-world V2X driving scenes with high fidelity; 2) using CRUISE improves 3D detection across ego-vehicle, infrastructure, and cooperative views, as well as cooperative 3D tracking on the V2X-Seq benchmark; and 3) CRUISE effectively generates challenging corner cases.