Gated Fields: Learning Scene Reconstruction from Gated Videos

TL;DR

Gated Fields is a neural scene reconstruction method that uses active gated videos to achieve accurate 3D scene reconstruction regardless of lighting conditions, outperforming traditional RGB and LiDAR approaches.

Contribution

The paper introduces Gated Fields, a novel neural reconstruction technique that leverages time-gated video data to improve scene geometry recovery in challenging lighting and texture conditions.

Findings

Achieves dense geometry reconstruction in day and night scenarios.

Outperforms RGB-based and LiDAR-based reconstruction methods.

Effective in poorly-lit and texture-deficient regions.

Abstract

Reconstructing outdoor 3D scenes from temporal observations is a challenge that recent work on neural fields has offered a new avenue for. However, existing methods that recover scene properties, such as geometry, appearance, or radiance, solely from RGB captures often fail when handling poorly-lit or texture-deficient regions. Similarly, recovering scenes with scanning LiDAR sensors is also difficult due to their low angular sampling rate which makes recovering expansive real-world scenes difficult. Tackling these gaps, we introduce Gated Fields - a neural scene reconstruction method that utilizes active gated video sequences. To this end, we propose a neural rendering approach that seamlessly incorporates time-gated capture and illumination. Our method exploits the intrinsic depth cues in the gated videos, achieving precise and dense geometry reconstruction irrespective of ambient illumination conditions. We validate the method across day and night scenarios and find that Gated Fields compares favorably to RGB and LiDAR reconstruction methods. Our code and datasets are available at https://light.princeton.edu/gatedfields/.