HyGE-Occ: Hybrid View-Transformation with 3D Gaussian and Edge Priors for 3D Panoptic Occupancy Prediction

TL;DR

HyGE-Occ introduces a hybrid view-transformation framework with 3D Gaussian and edge priors to improve geometric consistency and boundary accuracy in 3D panoptic occupancy prediction, leading to better scene understanding.

Contribution

The paper proposes a novel hybrid view-transformation approach combining Gaussian and discretized depth representations, along with edge priors, to enhance 3D geometric reasoning and boundary detection.

Findings

Outperforms existing methods on Occ3D-nuScenes dataset

Improves geometric consistency and structural coherence in 3D scene maps

Enhances boundary awareness through edge map auxiliary learning

Abstract

3D Panoptic Occupancy Prediction aims to reconstruct a dense volumetric scene map by predicting the semantic class and instance identity of every occupied region in 3D space. Achieving such fine-grained 3D understanding requires precise geometric reasoning and spatially consistent scene representation across complex environments. However, existing approaches often struggle to maintain precise geometry and capture the precise spatial range of 3D instances critical for robust panoptic separation. To overcome these limitations, we introduce HyGE-Occ, a novel framework that leverages a hybrid view-transformation branch with 3D Gaussian and edge priors to enhance both geometric consistency and boundary awareness in 3D panoptic occupancy prediction. HyGE-Occ employs a hybrid view-transformation branch that fuses a continuous Gaussian-based depth representation with a discretized depth-bin formulation, producing BEV features with improved geometric consistency and structural coherence. In parallel, we extract edge maps from BEV features and use them as auxiliary information to learn edge cues. In our extensive experiments on the Occ3D-nuScenes dataset, HyGE-Occ outperforms existing work, demonstrating superior 3D geometric reasoning.