MVPainter: Accurate and Detailed 3D Texture Generation via Multi-View Diffusion with Geometric Control

TL;DR

MVPainter introduces a novel multi-view diffusion approach with geometric control to generate accurate, detailed 3D textures that are aligned with geometry and suitable for real-world rendering.

Contribution

The paper presents MVPainter, a new framework that enhances 3D texture generation through data augmentation, geometric conditioning, and PBR attribute extraction, achieving state-of-the-art results.

Findings

State-of-the-art performance in texture fidelity and detail.

Improved alignment between textures and geometry.

Generated textures suitable for real-world rendering applications.

Abstract

Recently, significant advances have been made in 3D object generation. Building upon the generated geometry, current pipelines typically employ image diffusion models to generate multi-view RGB images, followed by UV texture reconstruction through texture baking. While 3D geometry generation has improved significantly, supported by multiple open-source frameworks, 3D texture generation remains underexplored. In this work, we systematically investigate 3D texture generation through the lens of three core dimensions: reference-texture alignment, geometry-texture consistency, and local texture quality. To tackle these issues, we propose MVPainter, which employs data filtering and augmentation strategies to enhance texture fidelity and detail, and introduces ControlNet-based geometric conditioning to improve texture-geometry alignment. Furthermore, we extract physically-based rendering (PBR) attributes from the generated views to produce PBR meshes suitable for real-world rendering applications. MVPainter achieves state-of-the-art results across all three dimensions, as demonstrated by human-aligned evaluations. To facilitate further research and reproducibility, we also release our full pipeline as an open-source system, including data construction, model architecture, and evaluation tools.