FlexPainter: Flexible and Multi-View Consistent Texture Generation

TL;DR

FlexPainter is a novel texture generation pipeline that offers flexible multi-modal guidance and ensures multi-view consistency, significantly improving the quality and control in 3D texture map creation using diffusion models.

Contribution

We introduce FlexPainter, a new method that combines a shared conditional embedding space with multi-view diffusion and adaptive synchronization to produce consistent, high-quality textures.

Findings

Outperforms state-of-the-art methods in flexibility and quality

Achieves highly consistent multi-view texture generation

Enables reference image-based stylization

Abstract

Texture map production is an important part of 3D modeling and determines the rendering quality. Recently, diffusion-based methods have opened a new way for texture generation. However, restricted control flexibility and limited prompt modalities may prevent creators from producing desired results. Furthermore, inconsistencies between generated multi-view images often lead to poor texture generation quality. To address these issues, we introduce \textbf{FlexPainter}, a novel texture generation pipeline that enables flexible multi-modal conditional guidance and achieves highly consistent texture generation. A shared conditional embedding space is constructed to perform flexible aggregation between different input modalities. Utilizing such embedding space, we present an image-based CFG method to decompose structural and style information, achieving reference image-based stylization. Leveraging the 3D knowledge within the image diffusion prior, we first generate multi-view images simultaneously using a grid representation to enhance global understanding. Meanwhile, we propose a view synchronization and adaptive weighting module during diffusion sampling to further ensure local consistency. Finally, a 3D-aware texture completion model combined with a texture enhancement model is used to generate seamless, high-resolution texture maps. Comprehensive experiments demonstrate that our framework significantly outperforms state-of-the-art methods in both flexibility and generation quality.