HeightFormer: Explicit Height Modeling without Extra Data for Camera-only 3D Object Detection in Bird's Eye View

TL;DR

HeightFormer introduces an explicit height modeling approach in BEV space for camera-only 3D object detection, achieving state-of-the-art results without requiring additional data like LiDAR.

Contribution

The paper proposes HeightFormer, a novel method that explicitly models heights in BEV space, providing a theoretical proof of its equivalence to depth-based methods and demonstrating superior performance.

Findings

Achieves state-of-the-art performance among camera-only methods

Models heights and uncertainties in a self-recursive manner

No extra data like LiDAR needed for accurate height estimation

Abstract

Vision-based Bird's Eye View (BEV) representation is an emerging perception formulation for autonomous driving. The core challenge is to construct BEV space with multi-camera features, which is a one-to-many ill-posed problem. Diving into all previous BEV representation generation methods, we found that most of them fall into two types: modeling depths in image views or modeling heights in the BEV space, mostly in an implicit way. In this work, we propose to explicitly model heights in the BEV space, which needs no extra data like LiDAR and can fit arbitrary camera rigs and types compared to modeling depths. Theoretically, we give proof of the equivalence between height-based methods and depth-based methods. Considering the equivalence and some advantages of modeling heights, we propose HeightFormer, which models heights and uncertainties in a self-recursive way. Without any extra data, the proposed HeightFormer could estimate heights in BEV accurately. Benchmark results show that the performance of HeightFormer achieves SOTA compared with those camera-only methods.