Texture Generation on 3D Meshes with Point-UV Diffusion

TL;DR

This paper introduces Point-UV diffusion, a novel coarse-to-fine pipeline combining diffusion models and UV mapping to synthesize high-quality, 3D consistent textures on meshes of any genus, improving texture fidelity and diversity.

Contribution

The work presents a new Point-UV diffusion pipeline that integrates point and UV diffusion models for high-quality, 3D consistent texture synthesis on arbitrary meshes, addressing previous limitations.

Findings

Successfully generates diversified textures with global consistency.

Handles meshes of any genus effectively.

Achieves high-fidelity, geometry-compatible textures.

Abstract

In this work, we focus on synthesizing high-quality textures on 3D meshes. We present Point-UV diffusion, a coarse-to-fine pipeline that marries the denoising diffusion model with UV mapping to generate 3D consistent and high-quality texture images in UV space. We start with introducing a point diffusion model to synthesize low-frequency texture components with our tailored style guidance to tackle the biased color distribution. The derived coarse texture offers global consistency and serves as a condition for the subsequent UV diffusion stage, aiding in regularizing the model to generate a 3D consistent UV texture image. Then, a UV diffusion model with hybrid conditions is developed to enhance the texture fidelity in the 2D UV space. Our method can process meshes of any genus, generating diversified, geometry-compatible, and high-fidelity textures. Code is available at https://cvmi-lab.github.io/Point-UV-Diffusion