Non-line-of-Sight Imaging via Neural Transient Fields

TL;DR

This paper introduces a neural transient field (NeTF) framework for non-line-of-sight imaging, enabling high-quality scene reconstruction without explicit geometry recovery, by modeling transient measurements with a neural network.

Contribution

The paper proposes a novel NeTF approach inspired by NeRF, using a spherical volume neural model and Monte Carlo sampling to improve NLOS imaging quality.

Findings

NeTF achieves higher quality reconstructions than state-of-the-art methods.

NeTF preserves fine scene details in synthetic and real datasets.

The spherical volume formulation is effective for both confocal and non-confocal setups.

Abstract

We present a neural modeling framework for Non-Line-of-Sight (NLOS) imaging. Previous solutions have sought to explicitly recover the 3D geometry (e.g., as point clouds) or voxel density (e.g., within a pre-defined volume) of the hidden scene. In contrast, inspired by the recent Neural Radiance Field (NeRF) approach, we use a multi-layer perceptron (MLP) to represent the neural transient field or NeTF. However, NeTF measures the transient over spherical wavefronts rather than the radiance along lines. We therefore formulate a spherical volume NeTF reconstruction pipeline, applicable to both confocal and non-confocal setups. Compared with NeRF, NeTF samples a much sparser set of viewpoints (scanning spots) and the sampling is highly uneven. We thus introduce a Monte Carlo technique to improve the robustness in the reconstruction. Comprehensive experiments on synthetic and real datasets demonstrate NeTF provides higher quality reconstruction and preserves fine details largely missing in the state-of-the-art.