Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Dynamic Scenes

TL;DR

The paper presents DG-Mesh, a novel framework that reconstructs high-quality, time-consistent 3D meshes from dynamic scenes using Gaussian splatting, enabling detailed and temporally coherent mesh reconstructions for graphics applications.

Contribution

Introduces DG-Mesh, a new method leveraging 3D Gaussian splatting and Gaussian-Mesh Anchoring for high-fidelity, time-consistent mesh reconstruction from dynamic observations.

Findings

DG-Mesh outperforms baselines in mesh reconstruction quality.

The method achieves better temporal consistency in dynamic scenes.

DG-Mesh enables applications like texture editing on moving objects.

Abstract

Modern 3D engines and graphics pipelines require mesh as a memory-efficient representation, which allows efficient rendering, geometry processing, texture editing, and many other downstream operations. However, it is still highly difficult to obtain high-quality mesh in terms of detailed structure and time consistency from dynamic observations. To this end, we introduce Dynamic Gaussians Mesh (DG-Mesh), a framework to reconstruct a high-fidelity and time-consistent mesh from dynamic input. Our work leverages the recent advancement in 3D Gaussian Splatting to construct the mesh sequence with temporal consistency from dynamic observations. Building on top of this representation, DG-Mesh recovers high-quality meshes from the Gaussian points and can track the mesh vertices over time, which enables applications such as texture editing on dynamic objects. We introduce the Gaussian-Mesh Anchoring, which encourages evenly distributed Gaussians, resulting better mesh reconstruction through mesh-guided densification and pruning on the deformed Gaussians. By applying cycle-consistent deformation between the canonical and the deformed space, we can project the anchored Gaussian back to the canonical space and optimize Gaussians across all time frames. During the evaluation on different datasets, DG-Mesh provides significantly better mesh reconstruction and rendering than baselines. Project page: https://www.liuisabella.com/DG-Mesh