Tessellation GS: Neural Mesh Gaussians for Robust Monocular Reconstruction of Dynamic Objects

TL;DR

Tessellation GS introduces a structured neural mesh Gaussian approach for robust, high-quality monocular dynamic scene reconstruction, overcoming limitations of previous Gaussian splatting methods in viewpoint extrapolation and dynamic object modeling.

Contribution

The paper presents Tessellation GS, a novel mesh-based neural Gaussian method that improves dynamic scene reconstruction from monocular views by hierarchical feature inference and adaptive Gaussian subdivision.

Findings

Reduces LPIPS by 29.1% compared to previous methods.

Decreases Chamfer distance by 49.2%, indicating more accurate reconstructions.

Enables robust reconstruction of dynamic objects from a single static camera.

Abstract

3D Gaussian Splatting (GS) enables highly photorealistic scene reconstruction from posed image sequences but struggles with viewpoint extrapolation due to its anisotropic nature, leading to overfitting and poor generalization, particularly in sparse-view and dynamic scene reconstruction. We propose Tessellation GS, a structured 2D GS approach anchored on mesh faces, to reconstruct dynamic scenes from a single continuously moving or static camera. Our method constrains 2D Gaussians to localized regions and infers their attributes via hierarchical neural features on mesh faces. Gaussian subdivision is guided by an adaptive face subdivision strategy driven by a detail-aware loss function. Additionally, we leverage priors from a reconstruction foundation model to initialize Gaussian deformations, enabling robust reconstruction of general dynamic objects from a single static camera, previously extremely challenging for optimization-based methods. Our method outperforms previous SOTA method, reducing LPIPS by 29.1% and Chamfer distance by 49.2% on appearance and mesh reconstruction tasks.