From Concept to Capability: Evaluating 3D Gaussian Splatting for Synthetic Scene Editing in Autonomous Driving

TL;DR

This paper evaluates the fidelity of 3D Gaussian Splatting in reconstructing safety-critical scenes for autonomous driving, focusing on vehicles and pedestrians, to support industrial deployment.

Contribution

It introduces a systematic framework for analyzing the reconstruction quality of 3D Gaussian Splatting in safety-related scenarios for autonomous driving.

Findings

Reconstruction fidelity degrades with multiple novel viewpoints.

Analysis highlights limitations in reconstructing occluded objects.

Provides insights for integrating 3DGS into industrial AD testing pipelines.

Abstract

The perception of an Autonomous Driving System (ADS) critically depends on relevant, comprehensive, and diverse datasets to ensure its safety while operating in the environment. Field data collection lacks completeness with respect to the list of rare but still possible safety-related scenarios needed for the development, verification, and validation of the ADS. 3D Gaussian Splatting (3DGS) has shown promising capabilities for the reconstruction and editing of scenes based on data collected by cameras and LiDAR sensors. However, the industrial fidelity evaluation of reconstructions is underexplored, which is crucial when employing such methods in safety-related systems, especially for ADS. This becomes more challenging as ADS operates in a dynamic, uncontrolled environment with limited viewpoints and often partially occluded objects. This paper addresses this gap by proposing and implementing a framework (Fig. 1) to systematically analyze the capabilities and limitations of 3DGS for use in the reconstruction of safety-related scenes. It focuses on the quality of reconstruction for vehicles and pedestrians, which are the two most critical object classes for ADS. Our findings provide industry insights into the fidelity degradation of reconstructions from multiple novel viewpoints, both lateral and longitudinal, enabling the integration of these methods into real-world industrial AD software development and testing pipelines.