Compensating Spatiotemporally Inconsistent Observations for Online Dynamic 3D Gaussian Splatting

TL;DR

This paper introduces a novel method to improve temporal consistency in online 3D scene reconstruction from streaming videos by compensating for observation inconsistencies caused by real-world noise.

Contribution

It proposes a technique that learns and subtracts errors to restore ideal observations, significantly enhancing temporal consistency and rendering quality in online dynamic scene reconstruction.

Findings

Enhanced temporal consistency in online reconstructions

Improved rendering quality across datasets

Effective error compensation method

Abstract

Online reconstruction of dynamic scenes is significant as it enables learning scenes from live-streaming video inputs, while existing offline dynamic reconstruction methods rely on recorded video inputs. However, previous online reconstruction approaches have primarily focused on efficiency and rendering quality, overlooking the temporal consistency of their results, which often contain noticeable artifacts in static regions. This paper identifies that errors such as noise in real-world recordings affect temporal inconsistency in online reconstruction. We propose a method that enhances temporal consistency in online reconstruction from observations with temporal inconsistency which is inevitable in cameras. We show that our method restores the ideal observation by subtracting the learned error. We demonstrate that applying our method to various baselines significantly enhances both temporal consistency and rendering quality across datasets. Code, video results, and checkpoints are available at https://bbangsik13.github.io/OR2.