InsTex: Indoor Scenes Stylized Texture Synthesis

TL;DR

InsTex is a novel two-stage method that leverages depth-to-image diffusion priors to generate high-quality, style-consistent textures for 3D indoor scenes, addressing generalization and efficiency issues in existing approaches.

Contribution

The paper introduces InsTex, a two-stage architecture that combines 2D diffusion models with depth priors for improved 3D scene texturing, achieving state-of-the-art results.

Findings

Outperforms existing methods in visual quality and consistency.

Supports both textual and visual prompts for flexible texturing.

Effective across various 3D indoor scene applications.

Abstract

Generating high-quality textures for 3D scenes is crucial for applications in interior design, gaming, and augmented/virtual reality (AR/VR). Although recent advancements in 3D generative models have enhanced content creation, significant challenges remain in achieving broad generalization and maintaining style consistency across multiple viewpoints. Current methods, such as 2D diffusion models adapted for 3D texturing, suffer from lengthy processing times and visual artifacts, while approaches driven by 3D data often fail to generalize effectively. To overcome these challenges, we introduce InsTex, a two-stage architecture designed to generate high-quality, style-consistent textures for 3D indoor scenes. InsTex utilizes depth-to-image diffusion priors in a coarse-to-fine pipeline, first generating multi-view images with a pre-trained 2D diffusion model and subsequently refining the textures for consistency. Our method supports both textual and visual prompts, achieving state-of-the-art results in visual quality and quantitative metrics, and demonstrates its effectiveness across various 3D texturing applications.