CVCP-Fusion: On Implicit Depth Estimation for 3D Bounding Box Prediction

TL;DR

This paper introduces CVCP-Fusion, a novel 3D object detection model that combines camera and LiDAR data in BEV space to improve 3D bounding box accuracy while maintaining real-time efficiency.

Contribution

It proposes a new fusion architecture that preserves semantic information from camera data and spatial data from LiDAR, integrating established algorithms for enhanced 3D detection.

Findings

Implicit depth estimation suffices for 2D map-view tasks.

Explicit geometric information improves 3D bounding box accuracy.

The model achieves real-time processing capabilities.

Abstract

Combining LiDAR and Camera-view data has become a common approach for 3D Object Detection. However, previous approaches combine the two input streams at a point-level, throwing away semantic information derived from camera features. In this paper we propose Cross-View Center Point-Fusion, a state-of-the-art model to perform 3D object detection by combining camera and LiDAR-derived features in the BEV space to preserve semantic density from the camera stream while incorporating spacial data from the LiDAR stream. Our architecture utilizes aspects from previously established algorithms, Cross-View Transformers and CenterPoint, and runs their backbones in parallel, allowing efficient computation for real-time processing and application. In this paper we find that while an implicitly calculated depth-estimate may be sufficiently accurate in a 2D map-view representation, explicitly calculated geometric and spacial information is needed for precise bounding box prediction in the 3D world-view space.