Lightweight Deep Learning Architecture for MPI Correction and Transient Reconstruction

TL;DR

This paper introduces a compact deep learning model that effectively corrects Multi-Path Interference in iToF cameras and reconstructs transient information, achieving state-of-the-art results and robustness to noise.

Contribution

A novel lightweight architecture that directly models transient light propagation for MPI correction and transient reconstruction in iToF data.

Findings

Achieves state-of-the-art MPI correction on synthetic and real data.

Performs robustly under extreme noise conditions.

Advances transient reconstruction from multi-frequency iToF data.

Abstract

Indirect Time-of-Flight cameras (iToF) are low-cost devices that provide depth images at an interactive frame rate. However, they are affected by different error sources, with the spotlight taken by Multi-Path Interference (MPI), a key challenge for this technology. Common data-driven approaches tend to focus on a direct estimation of the output depth values, ignoring the underlying transient propagation of the light in the scene. In this work instead, we propose a very compact architecture, leveraging on the direct-global subdivision of transient information for the removal of MPI and for the reconstruction of the transient information itself. The proposed model reaches state-of-the-art MPI correction performances both on synthetic and real data and proves to be very competitive also at extreme levels of noise; at the same time, it also makes a step towards reconstructing transient information from multi-frequency iToF data.