HR Human: Modeling Human Avatars with Triangular Mesh and High-Resolution Textures from Videos

TL;DR

This paper introduces a novel framework for creating high-resolution, editable human avatars from monocular videos, combining neural implicit surfaces with explicit triangular meshes and physically-based textures for better integration with traditional graphics engines.

Contribution

The method uniquely fuses monocular video data to generate deformable neural implicit surfaces with explicit triangular meshes and high-resolution textures, enabling more editable and realistic human avatars.

Findings

Outperforms previous methods in fidelity and realism.

Supports deployment on common rendering engines.

Provides high-resolution, physically-based textures for avatars.

Abstract

Recently, implicit neural representation has been widely used to generate animatable human avatars. However, the materials and geometry of those representations are coupled in the neural network and hard to edit, which hinders their application in traditional graphics engines. We present a framework for acquiring human avatars that are attached with high-resolution physically-based material textures and triangular mesh from monocular video. Our method introduces a novel information fusion strategy to combine the information from the monocular video and synthesize virtual multi-view images to tackle the sparsity of the input view. We reconstruct humans as deformable neural implicit surfaces and extract triangle mesh in a well-behaved pose as the initial mesh of the next stage. In addition, we introduce an approach to correct the bias for the boundary and size of the coarse mesh extracted. Finally, we adapt prior knowledge of the latent diffusion model at super-resolution in multi-view to distill the decomposed texture. Experiments show that our approach outperforms previous representations in terms of high fidelity, and this explicit result supports deployment on common renderers.