VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention

TL;DR

VISTA introduces a novel multi-view fusion module with dual cross-view spatial attention for improved 3D object detection from LiDAR data, significantly enhancing accuracy on autonomous driving benchmarks.

Contribution

The paper proposes a new plug-and-play fusion module with a convolutional attention mechanism and decoupled tasks, advancing multi-view 3D detection performance.

Findings

Achieves 63.0% mAP and 69.8% NDS on nuScenes

Outperforms existing methods by up to 24% in key categories

Demonstrates effectiveness through extensive experiments

Abstract

Detecting objects from LiDAR point clouds is of tremendous significance in autonomous driving. In spite of good progress, accurate and reliable 3D detection is yet to be achieved due to the sparsity and irregularity of LiDAR point clouds. Among existing strategies, multi-view methods have shown great promise by leveraging the more comprehensive information from both bird's eye view (BEV) and range view (RV). These multi-view methods either refine the proposals predicted from single view via fused features, or fuse the features without considering the global spatial context; their performance is limited consequently. In this paper, we propose to adaptively fuse multi-view features in a global spatial context via Dual Cross-VIew SpaTial Attention (VISTA). The proposed VISTA is a novel plug-and-play fusion module, wherein the multi-layer perceptron widely adopted in standard attention modules is replaced with a convolutional one. Thanks to the learned attention mechanism, VISTA can produce fused features of high quality for prediction of proposals. We decouple the classification and regression tasks in VISTA, and an additional constraint of attention variance is applied that enables the attention module to focus on specific targets instead of generic points. We conduct thorough experiments on the benchmarks of nuScenes and Waymo; results confirm the efficacy of our designs. At the time of submission, our method achieves 63.0% in overall mAP and 69.8% in NDS on the nuScenes benchmark, outperforming all published methods by up to 24% in safety-crucial categories such as cyclist. The source code in PyTorch is available at https://github.com/Gorilla-Lab-SCUT/VISTA