Automatic calibration of time of flight based non-line-of-sight reconstruction

TL;DR

This paper introduces an automatic calibration method for NLOS imaging that jointly reconstructs the scene and calibrates system parameters, improving robustness against calibration errors using a differentiable forward model and gradient descent.

Contribution

It presents a novel differentiable forward model enabling joint scene reconstruction and calibration parameter recovery without regularization.

Findings

Robust reconstructions achieved with simulated data despite calibration errors.

Method outperforms state-of-the-art algorithms under mis-calibration conditions.

Effective on real data with calibration inaccuracies.

Abstract

Time of flight based Non-line-of-sight (NLOS) imaging approaches require precise calibration of illumination and detector positions on the visible scene to produce reasonable results. If this calibration error is sufficiently high, reconstruction can fail entirely without any indication to the user. In this work, we highlight the necessity of building autocalibration into NLOS reconstruction in order to handle mis-calibration. We propose a forward model of NLOS measurements that is differentiable with respect to both, the hidden scene albedo, and virtual illumination and detector positions. With only a mean squared error loss and no regularization, our model enables joint reconstruction and recovery of calibration parameters by minimizing the measurement residual using gradient descent. We demonstrate our method is able to produce robust reconstructions using simulated and real data where the calibration error applied causes other state of the art algorithms to fail.