Detailed Avatar Recovery from Single Image

TL;DR

This paper introduces a learning-based framework that reconstructs detailed 3D human avatars from a single image, capturing surface details and textures beyond traditional parametric models.

Contribution

It combines parametric models with free-form 3D deformation using neural networks within a Hierarchical Mesh Deformation framework for detailed avatar recovery.

Findings

Outperforms previous methods in accuracy

Achieves detailed surface and texture reconstruction

Effective in various poses and viewpoints

Abstract

This paper presents a novel framework to recover \emph{detailed} avatar from a single image. It is a challenging task due to factors such as variations in human shapes, body poses, texture, and viewpoints. Prior methods typically attempt to recover the human body shape using a parametric-based template that lacks the surface details. As such resulting body shape appears to be without clothing. In this paper, we propose a novel learning-based framework that combines the robustness of the parametric model with the flexibility of free-form 3D deformation. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, utilizing the constraints from body joints, silhouettes, and per-pixel shading information. Our method can restore detailed human body shapes with complete textures beyond skinned models. Experiments demonstrate that our method has outperformed previous state-of-the-art approaches, achieving better accuracy in terms of both 2D IoU number and 3D metric distance.