FreeTimeGS: Free Gaussian Primitives at Anytime and Anywhere for Dynamic Scene Reconstruction

TL;DR

FreeTimeGS introduces a flexible 4D Gaussian primitive representation for dynamic scene reconstruction, enabling primitives to appear at arbitrary times and locations, significantly improving rendering quality in complex motion scenarios.

Contribution

The paper proposes a novel 4D Gaussian primitive model with motion functions, enhancing dynamic scene reconstruction beyond canonical approaches.

Findings

Outperforms recent methods in rendering quality

Handles complex motions more effectively

Reduces temporal redundancy in dynamic scenes

Abstract

This paper addresses the challenge of reconstructing dynamic 3D scenes with complex motions. Some recent works define 3D Gaussian primitives in the canonical space and use deformation fields to map canonical primitives to observation spaces, achieving real-time dynamic view synthesis. However, these methods often struggle to handle scenes with complex motions due to the difficulty of optimizing deformation fields. To overcome this problem, we propose FreeTimeGS, a novel 4D representation that allows Gaussian primitives to appear at arbitrary time and locations. In contrast to canonical Gaussian primitives, our representation possesses the strong flexibility, thus improving the ability to model dynamic 3D scenes. In addition, we endow each Gaussian primitive with an motion function, allowing it to move to neighboring regions over time, which reduces the temporal redundancy. Experiments results on several datasets show that the rendering quality of our method outperforms recent methods by a large margin. Project page: https://zju3dv.github.io/freetimegs/ .