S2GO: Streaming Sparse Gaussian Occupancy Prediction

TL;DR

S2GO introduces a streaming, query-based 3D occupancy prediction method that efficiently captures scene dynamics, outperforming dense voxel-based approaches in accuracy and speed on major benchmarks.

Contribution

It proposes a novel streaming sparse query framework for 3D occupancy prediction, improving efficiency and flexibility over traditional dense representations.

Findings

Achieves state-of-the-art performance on nuScenes and KITTI benchmarks.

Outperforms prior methods like GaussianWorld by 1.5 IoU.

Provides 5.9x faster inference than dense approaches.

Abstract

Despite the demonstrated efficiency and performance of sparse query-based representations for perception, state-of-the-art 3D occupancy prediction methods still rely on voxel-based or dense Gaussian-based 3D representations. However, dense representations are slow, and they lack flexibility in capturing the temporal dynamics of driving scenes. Distinct from prior work, we instead summarize the scene into a compact set of 3D queries which are propagated through time in an online, streaming fashion. These queries are then decoded into semantic Gaussians at each timestep. We couple our framework with a denoising rendering objective to guide the queries and their constituent Gaussians in effectively capturing scene geometry. Owing to its efficient, query-based representation, S2GO achieves state-of-the-art performance on the nuScenes and KITTI occupancy benchmarks, outperforming prior art (e.g., GaussianWorld) by 1.5 IoU with 5.9x faster inference.