DeMapGS: Simultaneous Mesh Deformation and Surface Attribute Mapping via Gaussian Splatting

TL;DR

DeMapGS introduces a unified Gaussian Splatting framework for deformable surface reconstruction and attribute mapping, improving editing flexibility and rendering quality over prior independent point-based methods.

Contribution

It presents a novel joint optimization approach anchoring Gaussian splats to a deformable mesh, overcoming topological issues and enabling high-fidelity surface attribute extraction.

Findings

Achieves state-of-the-art mesh reconstruction quality.

Supports flexible editing and cross-object manipulation.

Enhances rendering realism with diffuse, normal, and displacement maps.

Abstract

We propose DeMapGS, a structured Gaussian Splatting framework that jointly optimizes deformable surfaces and surface-attached 2D Gaussian splats. By anchoring splats to a deformable template mesh, our method overcomes topological inconsistencies and enhances editing flexibility, addressing limitations of prior Gaussian Splatting methods that treat points independently. The unified representation in our method supports extraction of high-fidelity diffuse, normal, and displacement maps, enabling the reconstructed mesh to inherit the photorealistic rendering quality of Gaussian Splatting. To support robust optimization, we introduce a gradient diffusion strategy that propagates supervision across the surface, along with an alternating 2D/3D rendering scheme to handle concave regions. Experiments demonstrate that DeMapGS achieves state-of-the-art mesh reconstruction quality and enables downstream applications for Gaussian splats such as editing and cross-object manipulation through a shared parametric surface.