BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection

TL;DR

BEVNeXt revitalizes dense BEV frameworks for 3D object detection by integrating CRF-based depth estimation, temporal aggregation, and a two-stage decoder, achieving state-of-the-art results on nuScenes.

Contribution

The paper introduces BEVNeXt, a modernized dense BEV framework with novel components that enhance depth estimation and object localization, surpassing existing methods.

Findings

Achieves 64.2 NDS on nuScenes test set.

Outperforms both dense BEV and query-based frameworks.

Demonstrates superior depth estimation and localization accuracy.

Abstract

Recently, the rise of query-based Transformer decoders is reshaping camera-based 3D object detection. These query-based decoders are surpassing the traditional dense BEV (Bird's Eye View)-based methods. However, we argue that dense BEV frameworks remain important due to their outstanding abilities in depth estimation and object localization, depicting 3D scenes accurately and comprehensively. This paper aims to address the drawbacks of the existing dense BEV-based 3D object detectors by introducing our proposed enhanced components, including a CRF-modulated depth estimation module enforcing object-level consistencies, a long-term temporal aggregation module with extended receptive fields, and a two-stage object decoder combining perspective techniques with CRF-modulated depth embedding. These enhancements lead to a "modernized" dense BEV framework dubbed BEVNeXt. On the nuScenes benchmark, BEVNeXt outperforms both BEV-based and query-based frameworks under various settings, achieving a state-of-the-art result of 64.2 NDS on the nuScenes test set. Code will be available at \url{https://github.com/woxihuanjiangguo/BEVNeXt}.