BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidance

TL;DR

BEVDiffuser is a plug-and-play diffusion model that denoises BEV feature maps with ground-truth guidance, significantly improving 3D object detection performance in autonomous driving scenarios without extra computational costs.

Contribution

It introduces a novel diffusion-based denoising method for BEV representations that can be integrated into existing models without architectural changes.

Findings

Achieves 12.3% mAP improvement on nuScenes

Enhances long-tail object detection under challenging conditions

Improves BEV quality without additional computational overhead

Abstract

Bird's-eye-view (BEV) representations play a crucial role in autonomous driving tasks. Despite recent advancements in BEV generation, inherent noise, stemming from sensor limitations and the learning process, remains largely unaddressed, resulting in suboptimal BEV representations that adversely impact the performance of downstream tasks. To address this, we propose BEVDiffuser, a novel diffusion model that effectively denoises BEV feature maps using the ground-truth object layout as guidance. BEVDiffuser can be operated in a plug-and-play manner during training time to enhance existing BEV models without requiring any architectural modifications. Extensive experiments on the challenging nuScenes dataset demonstrate BEVDiffuser's exceptional denoising and generation capabilities, which enable significant enhancement to existing BEV models, as evidenced by notable improvements of 12.3\% in mAP and 10.1\% in NDS achieved for 3D object detection without introducing additional computational complexity. Moreover, substantial improvements in long-tail object detection and under challenging weather and lighting conditions further validate BEVDiffuser's effectiveness in denoising and enhancing BEV representations.