OG-Gaussian: Occupancy Based Street Gaussians for Autonomous Driving

TL;DR

OG-Gaussian introduces a novel method for 3D scene reconstruction in autonomous driving that uses camera-based occupancy grids and learning-based dynamic object tracking, reducing reliance on expensive sensors and annotations.

Contribution

The paper presents OG-Gaussian, a new approach that replaces LiDAR with camera-derived occupancy grids and employs learning-based methods for dynamic object reconstruction and tracking.

Findings

Achieves state-of-the-art reconstruction quality with PSNR of 35.13.

Runs at 143 FPS, enabling real-time applications.

Reduces computational costs compared to LiDAR-based methods.

Abstract

Accurate and realistic 3D scene reconstruction enables the lifelike creation of autonomous driving simulation environments. With advancements in 3D Gaussian Splatting (3DGS), previous studies have applied it to reconstruct complex dynamic driving scenes. These methods typically require expensive LiDAR sensors and pre-annotated datasets of dynamic objects. To address these challenges, we propose OG-Gaussian, a novel approach that replaces LiDAR point clouds with Occupancy Grids (OGs) generated from surround-view camera images using Occupancy Prediction Network (ONet). Our method leverages the semantic information in OGs to separate dynamic vehicles from static street background, converting these grids into two distinct sets of initial point clouds for reconstructing both static and dynamic objects. Additionally, we estimate the trajectories and poses of dynamic objects through a learning-based approach, eliminating the need for complex manual annotations. Experiments on Waymo Open dataset demonstrate that OG-Gaussian is on par with the current state-of-the-art in terms of reconstruction quality and rendering speed, achieving an average PSNR of 35.13 and a rendering speed of 143 FPS, while significantly reducing computational costs and economic overhead.