Accurate and Real-time Pseudo Lidar Detection: Is Stereo Neural Network Really Necessary?

TL;DR

This paper demonstrates that fast, less accurate stereo matching algorithms, combined with proper data refinement, can achieve real-time 3D detection comparable to state-of-the-art methods, challenging the necessity of complex neural networks.

Contribution

It reveals that high-precision stereo depth estimation is not essential for effective Pseudo-Lidar detection, enabling faster, low-latency 3D detection systems.

Findings

Achieves competitive accuracy with only 23 ms processing time.

Utilizes simple stereo matching with refinement to match state-of-the-art performance.

Supports real-time deployment in autonomous driving applications.

Abstract

The proposal of Pseudo-Lidar representation has significantly narrowed the gap between visual-based and active Lidar-based 3D object detection. However, current researches exclusively focus on pushing the accuracy improvement of Pseudo-Lidar by taking the advantage of complex and time-consuming neural networks. Seldom explore the profound characteristics of Pseudo-Lidar representation to obtain the promoting opportunities. In this paper, we dive deep into the pseudo Lidar representation and argue that the performance of 3D object detection is not fully dependent on the high precision stereo depth estimation. We demonstrate that even for the unreliable depth estimation, with proper data processing and refining, it can achieve comparable 3D object detection accuracy. With this finding, we further show the possibility that utilizing fast but inaccurate stereo matching algorithms in the Pseudo-Lidar system to achieve low latency responsiveness. In the experiments, we develop a system with a less powerful stereo matching predictor and adopt the proposed refinement schemes to improve the accuracy. The evaluation on the KITTI benchmark shows that the presented system achieves competitive accuracy to the state-of-the-art approaches with only 23 ms computing, showing it is a suitable candidate for deploying to real car-hold applications.