GSRender: Deduplicated Occupancy Prediction via Weakly Supervised 3D Gaussian Splatting

TL;DR

GSRender introduces a novel weakly-supervised 3D occupancy prediction method using Gaussian Splatting and innovative modules to reduce duplication and improve accuracy, achieving state-of-the-art results for autonomous driving perception.

Contribution

The paper presents GSRender, a new approach that simplifies sampling and reduces duplicated predictions in 3D occupancy perception using weak supervision and novel modules.

Findings

Achieves SOTA RayIoU (+6.0)

Reduces duplicated predictions significantly

Narrowing gap with 3D-supervised methods

Abstract

Weakly-supervised 3D occupancy perception is crucial for vision-based autonomous driving in outdoor environments. Previous methods based on NeRF often face a challenge in balancing the number of samples used. Too many samples can decrease efficiency, while too few can compromise accuracy, leading to variations in the mean Intersection over Union (mIoU) by 5-10 points. Furthermore, even with surrounding-view image inputs, only a single image is rendered from each viewpoint at any given moment. This limitation leads to duplicated predictions, which significantly impacts the practicality of the approach. However, this issue has largely been overlooked in existing research. To address this, we propose GSRender, which uses 3D Gaussian Splatting for weakly-supervised occupancy estimation, simplifying the sampling process. Additionally, we introduce the Ray Compensation module, which reduces duplicated predictions by compensating for features from adjacent frames. Finally, we redesign the dynamic loss to remove the influence of dynamic objects from adjacent frames. Extensive experiments show that our approach achieves SOTA results in RayIoU (+6.0), while also narrowing the gap with 3D- supervised methods. This work lays a solid foundation for weakly-supervised occupancy perception. The code is available at https://github.com/Jasper-sudo-Sun/GSRender.