AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction

TL;DR

AutoSplat introduces a novel Gaussian splatting framework with geometric and appearance constraints to enhance realistic scene reconstruction and view synthesis in autonomous driving, effectively handling complex backgrounds and dynamic objects.

Contribution

The paper presents AutoSplat, a new method that applies geometric constraints and residual spherical harmonics to improve scene reconstruction in autonomous driving scenarios.

Findings

Outperforms state-of-the-art methods on Pandaset and KITTI datasets.

Achieves highly realistic reconstructions with multi-view consistency.

Effectively models dynamic foreground objects and complex backgrounds.

Abstract

Realistic scene reconstruction and view synthesis are essential for advancing autonomous driving systems by simulating safety-critical scenarios. 3D Gaussian Splatting excels in real-time rendering and static scene reconstructions but struggles with modeling driving scenarios due to complex backgrounds, dynamic objects, and sparse views. We propose AutoSplat, a framework employing Gaussian splatting to achieve highly realistic reconstructions of autonomous driving scenes. By imposing geometric constraints on Gaussians representing the road and sky regions, our method enables multi-view consistent simulation of challenging scenarios including lane changes. Leveraging 3D templates, we introduce a reflected Gaussian consistency constraint to supervise both the visible and unseen side of foreground objects. Moreover, to model the dynamic appearance of foreground objects, we estimate residual spherical harmonics for each foreground Gaussian. Extensive experiments on Pandaset and KITTI demonstrate that AutoSplat outperforms state-of-the-art methods in scene reconstruction and novel view synthesis across diverse driving scenarios. Visit our project page at https://autosplat.github.io/.