RC-BEVFusion: A Plug-In Module for Radar-Camera Bird's Eye View Feature Fusion
Lukas St\"acker, Shashank Mishra, Philipp Heidenreich, Jason Rambach,, Didier Stricker

TL;DR
This paper introduces RC-BEVFusion, a modular neural network for radar-camera fusion on the bird's eye view plane, significantly improving 3D object detection performance in autonomous driving.
Contribution
It proposes BEVFeatureNet, a novel radar encoder, and demonstrates its integration into existing architectures, achieving state-of-the-art results without extensive tuning.
Findings
Up to 28% increase in nuScenes detection score
Best published result in radar-camera fusion category
Effective fusion of sparse radar and dense camera data
Abstract
Radars and cameras belong to the most frequently used sensors for advanced driver assistance systems and automated driving research. However, there has been surprisingly little research on radar-camera fusion with neural networks. One of the reasons is a lack of large-scale automotive datasets with radar and unmasked camera data, with the exception of the nuScenes dataset. Another reason is the difficulty of effectively fusing the sparse radar point cloud on the bird's eye view (BEV) plane with the dense images on the perspective plane. The recent trend of camera-based 3D object detection using BEV features has enabled a new type of fusion, which is better suited for radars. In this work, we present RC-BEVFusion, a modular radar-camera fusion network on the BEV plane. We propose BEVFeatureNet, a novel radar encoder branch, and show that it can be incorporated into several…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Advanced Image Fusion Techniques · Infrared Target Detection Methodologies
