3D Lidar Mapping Relative Accuracy Automatic Evaluation Algorithm

TL;DR

This paper introduces an automatic, ground-truth-free algorithm for evaluating the relative accuracy of 3D lidar maps used in autonomous vehicles, utilizing ghosting detection to indirectly measure map precision.

Contribution

The proposed method provides an automatic, indirect accuracy evaluation technique for 3D lidar mapping without requiring ground truth data, improving efficiency and practicality.

Findings

Effectively detects inaccurate poses with errors over 0.1m

Automatically calculates the percentage of bad poses

Provides a reliable accuracy metric for 3D lidar maps

Abstract

HD (High Definition) map based on 3D lidar plays a vital role in autonomous vehicle localization, planning, decision-making, perception, etc. Many 3D lidar mapping technologies related to SLAM (Simultaneous Localization and Mapping) are used in HD map construction to ensure its high accuracy. To evaluate the accuracy of 3D lidar mapping, the most common methods use ground truth of poses to calculate the error between estimated poses and ground truth, however it's usually so difficult to get the ground truth of poses in the actual lidar mapping for autonomous vehicle. In this paper, we proposed a relative accuracy evaluation algorithm that can automatically evaluate the accuracy of HD map built by 3D lidar mapping without ground truth. A method for detecting the degree of ghosting in point cloud map quantitatively is designed to reflect the accuracy indirectly, which takes advantage of the principle of light traveling in a straight line and the fact that light can not penetrate opaque objects. Our experimental results confirm that the proposed evaluation algorithm can automatically and efficiently detect the bad poses whose accuracy are less than the set threshold such as 0.1m, then calculate the bad poses percentage P_bad in all estimated poses to obtain the final accuracy metric P_acc = 1 - P_bad.