FSD-BEV: Foreground Self-Distillation for Multi-view 3D Object Detection

TL;DR

FSD-BEV introduces a novel foreground self-distillation approach combined with point cloud intensification and multi-scale feature enhancement to improve multi-view 3D object detection performance without relying on pre-trained teacher models.

Contribution

The paper proposes a self-distillation scheme that avoids distribution discrepancies, along with point cloud intensification and multi-scale foreground enhancement, advancing multi-view 3D detection methods.

Findings

Achieves state-of-the-art results on nuScenes dataset.

Effectively improves detection accuracy without pre-trained teachers.

Demonstrates robustness across various detection scenarios.

Abstract

Although multi-view 3D object detection based on the Bird's-Eye-View (BEV) paradigm has garnered widespread attention as an economical and deployment-friendly perception solution for autonomous driving, there is still a performance gap compared to LiDAR-based methods. In recent years, several cross-modal distillation methods have been proposed to transfer beneficial information from teacher models to student models, with the aim of enhancing performance. However, these methods face challenges due to discrepancies in feature distribution originating from different data modalities and network structures, making knowledge transfer exceptionally challenging. In this paper, we propose a Foreground Self-Distillation (FSD) scheme that effectively avoids the issue of distribution discrepancies, maintaining remarkable distillation effects without the need for pre-trained teacher models or cumbersome distillation strategies. Additionally, we design two Point Cloud Intensification (PCI) strategies to compensate for the sparsity of point clouds by frame combination and pseudo point assignment. Finally, we develop a Multi-Scale Foreground Enhancement (MSFE) module to extract and fuse multi-scale foreground features by predicted elliptical Gaussian heatmap, further improving the model's performance. We integrate all the above innovations into a unified framework named FSD-BEV. Extensive experiments on the nuScenes dataset exhibit that FSD-BEV achieves state-of-the-art performance, highlighting its effectiveness. The code and models are available at: https://github.com/CocoBoom/fsd-bev.