GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flow

TL;DR

GaussianFlowOcc introduces a sparse Gaussian-based approach for occupancy estimation that leverages temporal flow and weak supervision, achieving superior accuracy and significantly faster inference compared to previous methods.

Contribution

It presents a novel Gaussian-based sparse representation and a Gaussian Transformer architecture for efficient, weakly supervised occupancy estimation in 3D scenes.

Findings

Outperforms previous methods on nuScenes dataset

Achieves 50x faster inference speed

Effectively captures scene dynamics with temporal flow

Abstract

Occupancy estimation has become a prominent task in 3D computer vision, particularly within the autonomous driving community. In this paper, we present a novel approach to occupancy estimation, termed GaussianFlowOcc, which is inspired by Gaussian Splatting and replaces traditional dense voxel grids with a sparse 3D Gaussian representation. Our efficient model architecture based on a Gaussian Transformer significantly reduces computational and memory requirements by eliminating the need for expensive 3D convolutions used with inefficient voxel-based representations that predominantly represent empty 3D spaces. GaussianFlowOcc effectively captures scene dynamics by estimating temporal flow for each Gaussian during the overall network training process, offering a straightforward solution to a complex problem that is often neglected by existing methods. Moreover, GaussianFlowOcc is designed for scalability, as it employs weak supervision and does not require costly dense 3D voxel annotations based on additional data (e.g., LiDAR). Through extensive experimentation, we demonstrate that GaussianFlowOcc significantly outperforms all previous methods for weakly supervised occupancy estimation on the nuScenes dataset while featuring an inference speed that is 50 times faster than current SOTA.