GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detection

TL;DR

GeoBEV introduces a high-resolution BEV representation with novel sampling and labeling techniques to enhance multi-view 3D object detection accuracy, achieving state-of-the-art results on nuScenes.

Contribution

The paper proposes Radial-Cartesian BEV Sampling and In-Box Labeling to improve geometric fidelity in BEV representations for 3D detection.

Findings

Outperforms existing methods in BEV resolution and geometric accuracy

Achieves 66.2% NDS on nuScenes test set

Introduces novel loss and sampling strategies for better 3D scene understanding

Abstract

Bird's-Eye-View (BEV) representation has emerged as a mainstream paradigm for multi-view 3D object detection, demonstrating impressive perceptual capabilities. However, existing methods overlook the geometric quality of BEV representation, leaving it in a low-resolution state and failing to restore the authentic geometric information of the scene. In this paper, we identify the drawbacks of previous approaches that limit the geometric quality of BEV representation and propose Radial-Cartesian BEV Sampling (RC-Sampling), which outperforms other feature transformation methods in efficiently generating high-resolution dense BEV representation to restore fine-grained geometric information. Additionally, we design a novel In-Box Label to substitute the traditional depth label generated from the LiDAR points. This label reflects the actual geometric structure of objects rather than just their surfaces, injecting real-world geometric information into the BEV representation. In conjunction with the In-Box Label, Centroid-Aware Inner Loss (CAI Loss) is developed to capture the inner geometric structure of objects. Finally, we integrate the aforementioned modules into a novel multi-view 3D object detector, dubbed GeoBEV, which achieves a state-of-the-art result of 66.2\% NDS on the nuScenes test set. The code is available at https://github.com/mengtan00/GeoBEV.git.