Neural Parameterization for Dynamic Human Head Editing

TL;DR

Neural Parameterization (NeP) combines implicit and explicit scene representations to enable photo-realistic rendering and fine-grained, intuitive editing of dynamic human heads, including facial features, hair, and eyes.

Contribution

NeP introduces a hybrid scene representation that disentangles geometry and appearance, allowing both high-quality rendering and explicit, controllable editing of dynamic human heads.

Findings

Achieves comparable rendering accuracy to existing methods.

Enables intuitive geometric editing via sparse key points.

Maintains high detail in dynamic facial features.

Abstract

Implicit radiance functions emerged as a powerful scene representation for reconstructing and rendering photo-realistic views of a 3D scene. These representations, however, suffer from poor editability. On the other hand, explicit representations such as polygonal meshes allow easy editing but are not as suitable for reconstructing accurate details in dynamic human heads, such as fine facial features, hair, teeth, and eyes. In this work, we present Neural Parameterization (NeP), a hybrid representation that provides the advantages of both implicit and explicit methods. NeP is capable of photo-realistic rendering while allowing fine-grained editing of the scene geometry and appearance. We first disentangle the geometry and appearance by parameterizing the 3D geometry into 2D texture space. We enable geometric editability by introducing an explicit linear deformation blending layer. The deformation is controlled by a set of sparse key points, which can be explicitly and intuitively displaced to edit the geometry. For appearance, we develop a hybrid 2D texture consisting of an explicit texture map for easy editing and implicit view and time-dependent residuals to model temporal and view variations. We compare our method to several reconstruction and editing baselines. The results show that the NeP achieves almost the same level of rendering accuracy while maintaining high editability.