Robust and accurate depth estimation by fusing LiDAR and Stereo

TL;DR

This paper presents a fusion method combining LiDAR and stereo camera data to improve depth estimation accuracy and robustness, especially under challenging lighting and texture conditions, validated on the KITTI benchmark.

Contribution

The proposed approach effectively fuses LiDAR and stereo data, enhancing depth accuracy and robustness over traditional stereo matching methods.

Findings

Higher accuracy than classic methods on KITTI benchmark

Less affected by object texture and lighting variations

Improved speed due to disparity map up-sampling

Abstract

Depth estimation is one of the key technologies in some fields such as autonomous driving and robot navigation. However, the traditional method of using a single sensor is inevitably limited by the performance of the sensor. Therefore, a precision and robust method for fusing the LiDAR and stereo cameras is proposed. This method fully combines the advantages of the LiDAR and stereo camera, which can retain the advantages of the high precision of the LiDAR and the high resolution of images respectively. Compared with the traditional stereo matching method, the texture of the object and lighting conditions have less influence on the algorithm. Firstly, the depth of the LiDAR data is converted to the disparity of the stereo camera. Because the density of the LiDAR data is relatively sparse on the y-axis, the converted disparity map is up-sampled using the interpolation method. Secondly, in order to make full use of the precise disparity map, the disparity map and stereo matching are fused to propagate the accurate disparity. Finally, the disparity map is converted to the depth map. Moreover, the converted disparity map can also increase the speed of the algorithm. We evaluate the proposed pipeline on the KITTI benchmark. The experiment demonstrates that our algorithm has higher accuracy than several classic methods.