RePLAy: Remove Projective LiDAR Depthmap Artifacts via Exploiting Epipolar Geometry

TL;DR

RePLAy is a novel, parameter-free method that leverages epipolar geometry to eliminate projective artifacts in LiDAR depthmaps, enhancing the accuracy of 3D perception in autonomous vehicles without requiring stereo images.

Contribution

It introduces an analytical approach using epipolar geometry to remove LiDAR depthmap artifacts, applicable to datasets lacking stereo camera setups.

Findings

Consistently improves state-of-the-art monocular depth estimators.

Enhances 3D object detection accuracy.

Applicable to various autonomous vehicle datasets.

Abstract

3D sensing is a fundamental task for Autonomous Vehicles. Its deployment often relies on aligned RGB cameras and LiDAR. Despite meticulous synchronization and calibration, systematic misalignment persists in LiDAR projected depthmap. This is due to the physical baseline distance between the two sensors. The artifact is often reflected as background LiDAR incorrectly projected onto the foreground, such as cars and pedestrians. The KITTI dataset uses stereo cameras as a heuristic solution to remove artifacts. However most AV datasets, including nuScenes, Waymo, and DDAD, lack stereo images, making the KITTI solution inapplicable. We propose RePLAy, a parameter-free analytical solution to remove the projective artifacts. We construct a binocular vision system between a hypothesized virtual LiDAR camera and the RGB camera. We then remove the projective artifacts by determining the epipolar occlusion with the proposed analytical solution. We show unanimous improvement in the State-of-The-Art (SoTA) monocular depth estimators and 3D object detectors with the artifacts-free depthmaps.