SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object Detection
Hongcheng Zhang, Liu Liang, Pengxin Zeng, Xiao Song, Zhe Wang

TL;DR
SparseLIF introduces a novel sparse multi-modality 3D object detector that leverages perspective priors, refined sampling, and uncertainty-aware fusion to achieve state-of-the-art results on nuScenes.
Contribution
The paper presents a fully sparse, end-to-end multi-modality 3D detector with three innovative components for improved performance and robustness.
Findings
Achieves state-of-the-art results on nuScenes dataset.
Outperforms all existing 3D object detectors.
Demonstrates robustness against sensor noise.
Abstract
Sparse 3D detectors have received significant attention since the query-based paradigm embraces low latency without explicit dense BEV feature construction. However, these detectors achieve worse performance than their dense counterparts. In this paper, we find the key to bridging the performance gap is to enhance the awareness of rich representations in two modalities. Here, we present a high-performance fully sparse detector for end-to-end multi-modality 3D object detection. The detector, termed SparseLIF, contains three key designs, which are (1) Perspective-Aware Query Generation (PAQG) to generate high-quality 3D queries with perspective priors, (2) RoI-Aware Sampling (RIAS) to further refine prior queries by sampling RoI features from each modality, (3) Uncertainty-Aware Fusion (UAF) to precisely quantify the uncertainty of each sensor modality and adaptively conduct final…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Robotics and Sensor-Based Localization · Infrared Target Detection Methodologies
MethodsSparse Evolutionary Training
