Text-guided High-definition Consistency Texture Model

TL;DR

This paper introduces HCTM, a novel method that generates high-definition, consistent textures for 3D meshes guided by text prompts, overcoming the resolution and consistency limitations of previous diffusion models.

Contribution

HCTM leverages a pre-trained depth-to-image diffusion model with parameter-efficient fine-tuning and multi-diffusion strategies to produce high-resolution, consistent textures from text prompts.

Findings

Successfully generates high-definition textures for 3D meshes

Achieves consistent textures across multiple viewpoints

Demonstrates promising results through experiments

Abstract

With the advent of depth-to-image diffusion models, text-guided generation, editing, and transfer of realistic textures are no longer difficult. However, due to the limitations of pre-trained diffusion models, they can only create low-resolution, inconsistent textures. To address this issue, we present the High-definition Consistency Texture Model (HCTM), a novel method that can generate high-definition and consistent textures for 3D meshes according to the text prompts. We achieve this by leveraging a pre-trained depth-to-image diffusion model to generate single viewpoint results based on the text prompt and a depth map. We fine-tune the diffusion model with Parameter-Efficient Fine-Tuning to quickly learn the style of the generated result, and apply the multi-diffusion strategy to produce high-resolution and consistent results from different viewpoints. Furthermore, we propose a strategy that prevents the appearance of noise on the textures caused by backpropagation. Our proposed approach has demonstrated promising results in generating high-definition and consistent textures for 3D meshes, as demonstrated through a series of experiments.