SA-Det3D: Self-Attention Based Context-Aware 3D Object Detection

TL;DR

SA-Det3D introduces self-attention mechanisms into 3D object detection models, enhancing accuracy and efficiency by modeling long-range context in point-cloud data, applicable across various detectors and datasets.

Contribution

The paper proposes novel self-attention variants for 3D detection, improving performance and efficiency of existing detectors with minimal additional computational cost.

Findings

Up to 1.5% improvement in 3D AP on KITTI

Parameter reduction of 15-80% in models

Enhanced detection performance on KITTI, nuScenes, Waymo datasets

Abstract

Existing point-cloud based 3D object detectors use convolution-like operators to process information in a local neighbourhood with fixed-weight kernels and aggregate global context hierarchically. However, non-local neural networks and self-attention for 2D vision have shown that explicitly modeling long-range interactions can lead to more robust and competitive models. In this paper, we propose two variants of self-attention for contextual modeling in 3D object detection by augmenting convolutional features with self-attention features. We first incorporate the pairwise self-attention mechanism into the current state-of-the-art BEV, voxel and point-based detectors and show consistent improvement over strong baseline models of up to 1.5 3D AP while simultaneously reducing their parameter footprint and computational cost by 15-80% and 30-50%, respectively, on the KITTI validation set. We next propose a self-attention variant that samples a subset of the most representative features by learning deformations over randomly sampled locations. This not only allows us to scale explicit global contextual modeling to larger point-clouds, but also leads to more discriminative and informative feature descriptors. Our method can be flexibly applied to most state-of-the-art detectors with increased accuracy and parameter and compute efficiency. We show our proposed method improves 3D object detection performance on KITTI, nuScenes and Waymo Open datasets. Code is available at https://github.com/AutoVision-cloud/SA-Det3D.