Virtual Light Transport Matrices for Non-Line-Of-Sight Imaging

TL;DR

This paper presents a novel method to estimate light transport matrices in non-line-of-sight imaging, enabling detailed scene illumination and separation of various light components in hidden scenes.

Contribution

It introduces a framework coupling virtual light propagation models with LOS equations to estimate NLOS light transport matrices using specialized projector-camera setups.

Findings

Enables reillumination of hidden scene locations.

Separates direct and indirect light components.

Works under challenging wide-aperture NLOS conditions.

Abstract

The light transport matrix (LTM) is an instrumental tool in line-of-sight (LOS) imaging, describing how light interacts with the scene and enabling applications such as relighting or separation of illumination components. We introduce a framework to estimate the LTM of non-line-of-sight (NLOS) scenarios, coupling recent virtual forward light propagation models for NLOS imaging with the LOS light transport equation. We design computational projector-camera setups, and use these virtual imaging systems to estimate the transport matrix of hidden scenes. We introduce the specific illumination functions to compute the different elements of the matrix, overcoming the challenging wide-aperture conditions of NLOS setups. Our NLOS light transport matrix allows us to (re)illuminate specific locations of a hidden scene, and separate direct, first-order indirect, and higher-order indirect illumination of complex cluttered hidden scenes, similar to existing LOS techniques.