3D Lidar Reconstruction with Probabilistic Depth Completion for Robotic Navigation

TL;DR

This paper introduces a learning-based 3D lidar reconstruction method that densifies sparse lidar data using camera images, improving free space estimation for safer robotic navigation in outdoor environments.

Contribution

It presents a novel neural network framework that combines lidar and camera data to produce dense depth maps with uncertainty estimates, enhancing environment modeling for robotics.

Findings

Free space estimation increased by over 40% using the method.

The approach generalizes well from synthetic to real-world data without fine-tuning.

Denser reconstructions improve motion planning in outdoor scenes.

Abstract

Safe motion planning in robotics requires planning into space which has been verified to be free of obstacles. However, obtaining such environment representations using lidars is challenging by virtue of the sparsity of their depth measurements. We present a learning-aided 3D lidar reconstruction framework that upsamples sparse lidar depth measurements with the aid of overlapping camera images so as to generate denser reconstructions with more definitively free space than can be achieved with the raw lidar measurements alone. We use a neural network with an encoder-decoder structure to predict dense depth images along with depth uncertainty estimates which are fused using a volumetric mapping system. We conduct experiments on real-world outdoor datasets captured using a handheld sensing device and a legged robot. Using input data from a 16-beam lidar mapping a building network, our experiments showed that the amount of estimated free space was increased by more than 40% with our approach. We also show that our approach trained on a synthetic dataset generalises well to real-world outdoor scenes without additional fine-tuning. Finally, we demonstrate how motion planning tasks can benefit from these denser reconstructions.