Sketched RT3D: How to reconstruct billions of photons per second

TL;DR

This paper introduces a sketched reconstruction method for single-photon lidar that significantly reduces computational and memory demands, enabling real-time 3D scene reconstruction without sacrificing accuracy.

Contribution

It adapts existing algorithms to operate on small sketches of photon data, allowing efficient real-time 3D reconstruction at billions of photons per second.

Findings

Achieves comparable reconstruction quality with reduced data size

Significantly lowers execution time and memory usage

Enables real-time processing on high-rate lidar systems

Abstract

Single-photon light detection and ranging (lidar) captures depth and intensity information of a 3D scene. Reconstructing a scene from observed photons is a challenging task due to spurious detections associated with background illumination sources. To tackle this problem, there is a plethora of 3D reconstruction algorithms which exploit spatial regularity of natural scenes to provide stable reconstructions. However, most existing algorithms have computational and memory complexity proportional to the number of recorded photons. This complexity hinders their real-time deployment on modern lidar arrays which acquire billions of photons per second. Leveraging a recent lidar sketching framework, we show that it is possible to modify existing reconstruction algorithms such that they only require a small sketch of the photon information. In particular, we propose a sketched version of a recent state-of-the-art algorithm which uses point cloud denoisers to provide spatially regularized reconstructions. A series of experiments performed on real lidar datasets demonstrates a significant reduction of execution time and memory requirements, while achieving the same reconstruction performance than in the full data case.