HV-BEV: Decoupling Horizontal and Vertical Feature Sampling for Multi-View 3D Object Detection

TL;DR

HV-BEV introduces a novel decoupled feature sampling method for multi-view 3D object detection that separately handles horizontal aggregation and vertical height-aware sampling, improving detection accuracy especially for objects at different heights.

Contribution

The paper proposes HV-BEV, a new approach that decouples horizontal and vertical feature sampling in BEV-based 3D detection, enhancing object association and height awareness.

Findings

Achieves 50.5% mAP and 59.8% NDS on nuScenes test set.

Outperforms baseline methods in multi-view 3D detection tasks.

Effectively models height variation and object association across views.

Abstract

The application of vision-based multi-view environmental perception system has been increasingly recognized in autonomous driving technology, especially the BEV-based models. Current state-of-the-art solutions primarily encode image features from each camera view into the BEV space through explicit or implicit depth prediction. However, these methods often overlook the structured correlations among different parts of objects in 3D space and the fact that different categories of objects often occupy distinct local height ranges. For example, trucks appear at higher elevations, whereas traffic cones are near the ground. In this work, we propose a novel approach that decouples feature sampling in the \textbf{BEV} grid queries paradigm into \textbf{H}orizontal feature aggregation and \textbf{V}ertical adaptive height-aware reference point sampling (HV-BEV), aiming to improve both the aggregation of objects' complete information and awareness of diverse objects' height distribution. Specifically, a set of relevant neighboring points is dynamically constructed for each 3D reference point on the ground-aligned horizontal plane, enhancing the association of the same instance across different BEV grids, especially when the instance spans multiple image views around the vehicle. Additionally, instead of relying on uniform sampling within a fixed height range, we introduce a height-aware module that incorporates historical information, enabling the reference points to adaptively focus on the varying heights at which objects appear in different scenes. Extensive experiments validate the effectiveness of our proposed method, demonstrating its superior performance over the baseline across the nuScenes dataset. Moreover, our best-performing model achieves a remarkable 50.5\% mAP and 59.8\% NDS on the nuScenes testing set. The code is available at https://github.com/Uddd821/HV-BEV.