Meta 3D TextureGen: Fast and Consistent Texture Generation for 3D Objects

TL;DR

Meta 3D TextureGen is a fast, high-quality, and globally consistent texture generation method for 3D objects, leveraging text-to-image models conditioned on 3D semantics to produce detailed textures in under 20 seconds.

Contribution

It introduces a novel feedforward approach that combines 3D semantics with text-to-image models for rapid, high-resolution texture generation with global consistency.

Findings

Achieves state-of-the-art quality and speed in texture generation.

Produces 4K resolution textures through a novel up-scaling network.

Generates textures for complex geometries in less than 20 seconds.

Abstract

The recent availability and adaptability of text-to-image models has sparked a new era in many related domains that benefit from the learned text priors as well as high-quality and fast generation capabilities, one of which is texture generation for 3D objects. Although recent texture generation methods achieve impressive results by using text-to-image networks, the combination of global consistency, quality, and speed, which is crucial for advancing texture generation to real-world applications, remains elusive. To that end, we introduce Meta 3D TextureGen: a new feedforward method comprised of two sequential networks aimed at generating high-quality and globally consistent textures for arbitrary geometries of any complexity degree in less than 20 seconds. Our method achieves state-of-the-art results in quality and speed by conditioning a text-to-image model on 3D semantics in 2D space and fusing them into a complete and high-resolution UV texture map, as demonstrated by extensive qualitative and quantitative evaluations. In addition, we introduce a texture enhancement network that is capable of up-scaling any texture by an arbitrary ratio, producing 4k pixel resolution textures.