Blind Time-of-Flight Imaging: Sparse Deconvolution on the Continuum with Unknown Kernels

TL;DR

This paper presents a novel blind super-resolution technique for Time-of-Flight imaging that does not require kernel calibration, enabling high-resolution scene recovery across various modalities with practical hardware validation.

Contribution

It introduces a blind super-resolution method for ToF imaging that recovers sparse scenes on a continuum without prior kernel knowledge, leveraging approximation theory and alternating minimization.

Findings

Outperforms traditional calibration-based methods in accuracy and robustness.

Effective across multiple ToF modalities with hardware experiments.

Enhances super-resolution capabilities in challenging scenarios.

Abstract

In recent years, computational Time-of-Flight (ToF) imaging has emerged as an exciting and a novel imaging modality that offers new and powerful interpretations of natural scenes, with applications extending to 3D, light-in-flight, and non-line-of-sight imaging. Mathematically, ToF imaging relies on algorithmic super-resolution, as the back-scattered sparse light echoes lie on a finer time resolution than what digital devices can capture. Traditional methods necessitate knowledge of the emitted light pulses or kernels and employ sparse deconvolution to recover scenes. Unlike previous approaches, this paper introduces a novel, blind ToF imaging technique that does not require kernel calibration and recovers sparse spikes on a continuum, rather than a discrete grid. By studying the shared characteristics of various ToF modalities, we capitalize on the fact that most physical pulses approximately satisfy the Strang-Fix conditions from approximation theory. This leads to a new mathematical formulation for sparse super-resolution. Our recovery approach uses an optimization method that is pivoted on an alternating minimization strategy. We benchmark our blind ToF method against traditional kernel calibration methods, which serve as the baseline. Extensive hardware experiments across different ToF modalities demonstrate the algorithmic advantages, flexibility and empirical robustness of our approach. We show that our work facilitates super-resolution in scenarios where distinguishing between closely spaced objects is challenging, while maintaining performance comparable to known kernel situations. Examples of light-in-flight imaging and light-sweep videos highlight the practical benefits of our blind super-resolution method in enhancing the understanding of natural scenes.