Single-Photon 3D Imaging with Equi-Depth Photon Histograms

TL;DR

This paper introduces a novel 3D imaging method using equi-depth histograms to efficiently process single-photon data, enabling resource-constrained devices to perform high-resolution 3D vision tasks with lower bandwidth and memory requirements.

Contribution

It proposes an equi-depth histogram approach and an online estimation algorithm for single-photon cameras, reducing bandwidth and memory needs for 3D imaging in resource-limited settings.

Findings

ED histograms compress timestamp data more efficiently than EW histograms.

The online algorithm estimates ED histograms without storing all photon timestamps.

The method enables 3D scene understanding tasks under challenging conditions.

Abstract

Single-photon cameras present a promising avenue for high-resolution 3D imaging. They have ultra-high sensitivity -- down to individual photons -- and can record photon arrival times with extremely high (sub-nanosecond) resolution. Single-photon 3D cameras estimate the round-trip time of a laser pulse by forming equi-width (EW) histograms of detected photon timestamps. Acquiring and transferring such EW histograms requires high bandwidth and in-pixel memory, making SPCs less attractive in resource-constrained settings such as mobile devices and AR/VR headsets. In this work we propose a 3D sensing technique based on equi-depth (ED) histograms. ED histograms compress timestamp data more efficiently than EW histograms, reducing the bandwidth requirement. Moreover, to reduce the in-pixel memory requirement, we propose a lightweight algorithm to estimate ED histograms in an online fashion without explicitly storing the photon timestamps. This algorithm is amenable to future in-pixel implementations. We propose algorithms that process ED histograms to perform 3D computer-vision tasks of estimating scene distance maps and performing visual odometry under challenging conditions such as high ambient light. Our work paves the way towards lower bandwidth and reduced in-pixel memory requirements for SPCs, making them attractive for resource-constrained 3D vision applications. Project page: $\href{https://www.computational.camera/pedh}{https://www.computational.camera/pedh}$