AvatarTex: High-Fidelity Facial Texture Reconstruction from Single-Image Stylized Avatars

TL;DR

AvatarTex is a novel framework that combines diffusion models and GANs in a three-stage pipeline to generate high-fidelity, stylized, and photorealistic facial textures from a single image, overcoming dataset and consistency challenges.

Contribution

It introduces a diffusion-to-GAN pipeline and TexHub dataset, enabling high-quality multi-style facial texture reconstruction with geometric and artistic coherence.

Findings

Achieves state-of-the-art texture synthesis quality.

Effectively handles stylized and photorealistic textures.

Provides a new high-resolution multi-style texture dataset.

Abstract

We present AvatarTex, a high-fidelity facial texture reconstruction framework capable of generating both stylized and photorealistic textures from a single image. Existing methods struggle with stylized avatars due to the lack of diverse multi-style datasets and challenges in maintaining geometric consistency in non-standard textures. To address these limitations, AvatarTex introduces a novel three-stage diffusion-to-GAN pipeline. Our key insight is that while diffusion models excel at generating diversified textures, they lack explicit UV constraints, whereas GANs provide a well-structured latent space that ensures style and topology consistency. By integrating these strengths, AvatarTex achieves high-quality topology-aligned texture synthesis with both artistic and geometric coherence. Specifically, our three-stage pipeline first completes missing texture regions via diffusion-based inpainting, refines style and structure consistency using GAN-based latent optimization, and enhances fine details through diffusion-based repainting. To address the need for a stylized texture dataset, we introduce TexHub, a high-resolution collection of 20,000 multi-style UV textures with precise UV-aligned layouts. By leveraging TexHub and our structured diffusion-to-GAN pipeline, AvatarTex establishes a new state-of-the-art in multi-style facial texture reconstruction. TexHub will be released upon publication to facilitate future research in this field.