SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors

TL;DR

SceneTex introduces a diffusion-based method for high-quality, style-consistent texture synthesis in indoor scenes, leveraging a multiresolution texture field and cross-attention to improve visual fidelity and geometric accuracy.

Contribution

It formulates texture synthesis as an RGB optimization problem using diffusion priors, incorporating a multiresolution texture field and cross-attention decoder for enhanced consistency.

Findings

Significant improvements in visual quality over prior methods

Effective style and geometry consistency across views

High-fidelity texture generation for 3D indoor scenes

Abstract

We propose SceneTex, a novel method for effectively generating high-quality and style-consistent textures for indoor scenes using depth-to-image diffusion priors. Unlike previous methods that either iteratively warp 2D views onto a mesh surface or distillate diffusion latent features without accurate geometric and style cues, SceneTex formulates the texture synthesis task as an optimization problem in the RGB space where style and geometry consistency are properly reflected. At its core, SceneTex proposes a multiresolution texture field to implicitly encode the mesh appearance. We optimize the target texture via a score-distillation-based objective function in respective RGB renderings. To further secure the style consistency across views, we introduce a cross-attention decoder to predict the RGB values by cross-attending to the pre-sampled reference locations in each instance. SceneTex enables various and accurate texture synthesis for 3D-FRONT scenes, demonstrating significant improvements in visual quality and prompt fidelity over the prior texture generation methods.