UV-free Texture Generation with Denoising and Geodesic Heat Diffusions

TL;DR

This paper introduces a UV-free 3D surface texturing method using a surface-constrained denoising diffusion model on point-clouds, leveraging heat diffusion for spatial consistency and avoiding common UV-mapping issues.

Contribution

It proposes a novel UV-free texturing approach that employs heat diffusion and a diffusion probabilistic model on surface point-clouds, eliminating UV-related artifacts.

Findings

Effective UV-free texturing on 3D meshes

Long-distance texture consistency achieved

Code and models publicly available

Abstract

Seams, distortions, wasted UV space, vertex-duplication, and varying resolution over the surface are the most prominent issues of the standard UV-based texturing of meshes. These issues are particularly acute when automatic UV-unwrapping techniques are used. For this reason, instead of generating textures in automatically generated UV-planes like most state-of-the-art methods, we propose to represent textures as coloured point-clouds whose colours are generated by a denoising diffusion probabilistic model constrained to operate on the surface of 3D objects. Our sampling and resolution agnostic generative model heavily relies on heat diffusion over the surface of the meshes for spatial communication between points. To enable processing of arbitrarily sampled point-cloud textures and ensure long-distance texture consistency we introduce a fast re-sampling of the mesh spectral properties used during the heat diffusion and introduce a novel heat-diffusion-based self-attention mechanism. Our code and pre-trained models are available at github.com/simofoti/UV3-TeD.