TransBridge: Boost 3D Object Detection by Scene-Level Completion with Transformer Decoder

TL;DR

TransBridge enhances 3D object detection in autonomous driving by integrating scene-level completion with detection using a transformer-based up-sampling, improving accuracy especially in sparse regions without increasing costs.

Contribution

Introduces TransBridge, a transformer-based up-sampling block that fuses detection and completion features, and a dynamic-static reconstruction module for dense LiDAR data generation.

Findings

Improves end-to-end 3D detection mAP by 0.7 to 1.5 points.

Boosts two-stage detection mAP up to 5.78 points.

Demonstrates effectiveness on nuScenes and Waymo datasets.

Abstract

3D object detection is essential in autonomous driving, providing vital information about moving objects and obstacles. Detecting objects in distant regions with only a few LiDAR points is still a challenge, and numerous strategies have been developed to address point cloud sparsity through densification.This paper presents a joint completion and detection framework that improves the detection feature in sparse areas while maintaining costs unchanged. Specifically, we propose TransBridge, a novel transformer-based up-sampling block that fuses the features from the detection and completion networks.The detection network can benefit from acquiring implicit completion features derived from the completion network. Additionally, we design the Dynamic-Static Reconstruction (DSRecon) module to produce dense LiDAR data for the completion network, meeting the requirement for dense point cloud ground truth.Furthermore, we employ the transformer mechanism to establish connections between channels and spatial relations, resulting in a high-resolution feature map used for completion purposes.Extensive experiments on the nuScenes and Waymo datasets demonstrate the effectiveness of the proposed framework.The results show that our framework consistently improves end-to-end 3D object detection, with the mean average precision (mAP) ranging from 0.7 to 1.5 across multiple methods, indicating its generalization ability. For the two-stage detection framework, it also boosts the mAP up to 5.78 points.