Towards Accurate Single Panoramic 3D Detection: A Semantic Gaussian Centric Approach

TL;DR

This paper introduces PanoGSDet, a novel panoramic 3D detection framework using continuous semantic Gaussian representations to improve accuracy over existing grid-based methods.

Contribution

It proposes a monocular panoramic 3D detection approach that models features with continuous semantic Gaussians, enhancing geometric continuity and representation efficiency.

Findings

Outperforms existing methods on Structured3D dataset

Effectively models spherical features with semantic Gaussians

Refines 3D bounding boxes through Gaussian optimization

Abstract

Three-dimensional object detection in panoramic imagery is crucial for comprehensive scene understanding, yet accurately mapping 2D features to 3D remains a significant challenge. Prevailing methods often project 2D features onto discrete 3D grids, which break geometric continuity and limit representation efficiency. To overcome this limitation, this paper proposes PanoGSDet, a monocular panoramic 3D detection framework built upon continuous semantic 3D Gaussian representations. The proposed framework comprises a panoramic depth estimation component and a semantic Gaussian component. The panoramic depth estimation component extracts the equirectangular semantic and depth features from the monocular panorama input. The semantic Gaussian component includes a semantic Gaussian lifting module that projects spherical features into 3D semantic Gaussians, a semantic Gaussian optimization module that refines these semantic Gaussians, and a Gaussian guided prediction head that generates 3D bounding boxes from optimized Gaussian representations. Extensive experiments on the Structured3D dataset demonstrate that our method significantly outperforms existing methods.