Enhancing LiDAR Point Features with Foundation Model Priors for 3D Object Detection

TL;DR

This paper enhances LiDAR-based 3D object detection by integrating depth priors from foundation models, enriching point features, and employing a dual-path framework with bidirectional fusion, leading to improved detection accuracy on KITTI.

Contribution

It introduces a novel method to incorporate depth priors from foundation models into LiDAR point features and proposes a dual-path RoI framework with bidirectional fusion for better detection.

Findings

Consistent improvement in detection accuracy on KITTI benchmark.

Effective fusion of depth priors with LiDAR features enhances discriminative power.

The proposed framework outperforms existing methods in 3D object detection.

Abstract

Recent advances in foundation models have opened up new possibilities for enhancing 3D perception. In particular, DepthAnything offers dense and reliable geometric priors from monocular RGB images, which can complement sparse LiDAR data in autonomous driving scenarios. However, such priors remain underutilized in LiDAR-based 3D object detection. In this paper, we address the limited expressiveness of raw LiDAR point features, especially the weak discriminative capability of the reflectance attribute, by introducing depth priors predicted by DepthAnything. These priors are fused with the original LiDAR attributes to enrich each point's representation. To leverage the enhanced point features, we propose a point-wise feature extraction module. Then, a Dual-Path RoI feature extraction framework is employed, comprising a voxel-based branch for global semantic context and a point-based branch for fine-grained structural details. To effectively integrate the complementary RoI features, we introduce a bidirectional gated RoI feature fusion module that balances global and local cues. Extensive experiments on the KITTI benchmark show that our method consistently improves detection accuracy, demonstrating the value of incorporating visual foundation model priors into LiDAR-based 3D object detection.