RetimeGS: Continuous-Time Reconstruction of 4D Gaussian Splatting

TL;DR

RetimeGS introduces a continuous-time 4D Gaussian Splatting method that reduces ghosting artifacts and improves temporal coherence in dynamic scene rendering through explicit temporal modeling and optical flow guidance.

Contribution

It proposes a novel 4D Gaussian Splatting representation that explicitly models temporal behavior and employs optical flow techniques for ghost-free, coherent interpolation.

Findings

Outperforms state-of-the-art methods on datasets with fast motion and occlusions.

Achieves smooth, ghost-free rendering at arbitrary timestamps.

Effectively handles large motions and non-rigid deformations.

Abstract

Temporal retiming, the ability to reconstruct and render dynamic scenes at arbitrary timestamps, is crucial for applications such as slow-motion playback, temporal editing, and post-production. However, most existing 4D Gaussian Splatting (4DGS) methods overfit at discrete frame indices but struggle to represent continuous-time frames, leading to ghosting artifacts when interpolating between timestamps. We identify this limitation as a form of temporal aliasing and propose RetimeGS, a simple yet effective 4DGS representation that explicitly defines the temporal behavior of the 3D Gaussian and mitigates temporal aliasing. To achieve smooth and consistent interpolation, we incorporate optical flow-guided initialization and supervision, triple-rendering supervision, and other targeted strategies. Together, these components enable ghost-free, temporally coherent rendering even under large motions. Experiments on datasets featuring fast motion, non-rigid deformation, and severe occlusions demonstrate that RetimeGS achieves superior quality and coherence over state-of-the-art methods.