Generating Animatable 3D Cartoon Faces from Single Portraits

TL;DR

This paper introduces a novel framework that converts a single portrait into an animatable 3D cartoon face, combining style transfer, detailed reconstruction, and semantic rigging for realistic and expressive avatars.

Contribution

The paper presents a new method for generating detailed, animatable 3D cartoon faces from single images, integrating style transfer, a two-stage reconstruction, and semantic face rigging.

Findings

Achieves higher accuracy and aesthetic quality than previous methods.

Enables real-time facial animation of generated 3D models.

Successfully preserves identity and facial expressions in the 3D cartoon faces.

Abstract

With the booming of virtual reality (VR) technology, there is a growing need for customized 3D avatars. However, traditional methods for 3D avatar modeling are either time-consuming or fail to retain similarity to the person being modeled. We present a novel framework to generate animatable 3D cartoon faces from a single portrait image. We first transfer an input real-world portrait to a stylized cartoon image with a StyleGAN. Then we propose a two-stage reconstruction method to recover the 3D cartoon face with detailed texture, which first makes a coarse estimation based on template models, and then refines the model by non-rigid deformation under landmark supervision. Finally, we propose a semantic preserving face rigging method based on manually created templates and deformation transfer. Compared with prior arts, qualitative and quantitative results show that our method achieves better accuracy, aesthetics, and similarity criteria. Furthermore, we demonstrate the capability of real-time facial animation of our 3D model.