UVMap-ID: A Controllable and Personalized UV Map Generative Model

TL;DR

UVMap-ID is a novel controllable and personalized UV map generative model that fine-tunes a pre-trained diffusion model with face fusion and attribute data to produce high-quality, ID-specific textures.

Contribution

The paper introduces UVMap-ID, a new method for personalized UV map generation that combines fine-tuning of diffusion models with face fusion and a new evaluation framework.

Findings

Effective personalized UV map generation demonstrated.

Quantitative and qualitative results show high quality and controllability.

New metrics for UV map quality evaluation introduced.

Abstract

Recently, diffusion models have made significant strides in synthesizing realistic 2D human images based on provided text prompts. Building upon this, researchers have extended 2D text-to-image diffusion models into the 3D domain for generating human textures (UV Maps). However, some important problems about UV Map Generative models are still not solved, i.e., how to generate personalized texture maps for any given face image, and how to define and evaluate the quality of these generated texture maps. To solve the above problems, we introduce a novel method, UVMap-ID, which is a controllable and personalized UV Map generative model. Unlike traditional large-scale training methods in 2D, we propose to fine-tune a pre-trained text-to-image diffusion model which is integrated with a face fusion module for achieving ID-driven customized generation. To support the finetuning strategy, we introduce a small-scale attribute-balanced training dataset, including high-quality textures with labeled text and Face ID. Additionally, we introduce some metrics to evaluate the multiple aspects of the textures. Finally, both quantitative and qualitative analyses demonstrate the effectiveness of our method in controllable and personalized UV Map generation. Code is publicly available via https://github.com/twowwj/UVMap-ID.