Time of the Flight of the Gaussians: Optimizing Depth Indirectly in Dynamic Radiance Fields

TL;DR

This paper introduces a fast, high-fidelity method for reconstructing dynamic scenes from monocular C-ToF data, improving accuracy and speed over neural volumetric methods by optimizing Gaussian splatting with novel heuristics.

Contribution

It proposes a novel optimization approach with heuristics that enhances Gaussian splatting for dynamic scene reconstruction from C-ToF data, achieving 100x faster results.

Findings

Achieves comparable or better accuracy than neural volumetric methods.

Reconstructs fast-moving scenes like swinging baseball bats.

Operates efficiently with constrained C-ToF sensing conditions.

Abstract

We present a method to reconstruct dynamic scenes from monocular continuous-wave time-of-flight (C-ToF) cameras using raw sensor samples that achieves similar or better accuracy than neural volumetric approaches and is 100x faster. Quickly achieving high-fidelity dynamic 3D reconstruction from a single viewpoint is a significant challenge in computer vision. In C-ToF radiance field reconstruction, the property of interest-depth-is not directly measured, causing an additional challenge. This problem has a large and underappreciated impact upon the optimization when using a fast primitive-based scene representation like 3D Gaussian splatting, which is commonly used with multi-view data to produce satisfactory results and is brittle in its optimization otherwise. We incorporate two heuristics into the optimization to improve the accuracy of scene geometry represented by Gaussians. Experimental results show that our approach produces accurate reconstructions under constrained C-ToF sensing conditions, including for fast motions like swinging baseball bats. https://visual.cs.brown.edu/gftorf