Text-Guided Texturing by Synchronized Multi-View Diffusion

TL;DR

This paper presents a synchronized multi-view diffusion method for text-guided 3D object texturing, ensuring consistent and detailed textures by sharing information across views during the diffusion process.

Contribution

It introduces a novel synchronized diffusion approach that improves texture consistency by sharing denoised content among views during the diffusion process.

Findings

Achieves more consistent and seamless textures compared to previous methods.

Demonstrates superior performance in detail and realism of generated textures.

Ensures early consensus among views for better texture coherence.

Abstract

This paper introduces a novel approach to synthesize texture to dress up a given 3D object, given a text prompt. Based on the pretrained text-to-image (T2I) diffusion model, existing methods usually employ a project-and-inpaint approach, in which a view of the given object is first generated and warped to another view for inpainting. But it tends to generate inconsistent texture due to the asynchronous diffusion of multiple views. We believe such asynchronous diffusion and insufficient information sharing among views are the root causes of the inconsistent artifact. In this paper, we propose a synchronized multi-view diffusion approach that allows the diffusion processes from different views to reach a consensus of the generated content early in the process, and hence ensures the texture consistency. To synchronize the diffusion, we share the denoised content among different views in each denoising step, specifically blending the latent content in the texture domain from views with overlap. Our method demonstrates superior performance in generating consistent, seamless, highly detailed textures, comparing to state-of-the-art methods.