LumiTex: Towards High-Fidelity PBR Texture Generation with Illumination Context
Jingzhi Bao, Hongze Chen, Lingting Zhu, Chenyu Liu, Runze Zhang, Keyang Luo, Zeyu Hu, Weikai Chen, Yingda Yin, Xin Wang, Zehong Lin, Jun Zhang, Xiaoguang Han
TL;DR
LumiTex is an innovative framework that enhances PBR texture generation by disentangling material properties, incorporating illumination context, and ensuring seamless, view-consistent texture completion, leading to superior quality results.
Contribution
It introduces a novel multi-branch generation, lighting-aware attention, and geometry-guided inpainting to improve PBR texture synthesis from limited illumination cues.
Findings
Achieves state-of-the-art texture quality in experiments.
Outperforms existing open-source and commercial methods.
Ensures seamless and view-consistent texture completion.
Abstract
Physically-based rendering (PBR) provides a principled standard for realistic material-lighting interactions in computer graphics. Despite recent advances in generating PBR textures, existing methods fail to address two fundamental challenges: 1) materials decomposition from image prompts under limited illumination cues, and 2) seamless and view-consistent texture completion. To this end, we propose LumiTex, an end-to-end framework that comprises three key components: (1) a multi-branch generation scheme that disentangles albedo and metallic-roughness under shared illumination priors for robust material understanding, (2) a lighting-aware material attention mechanism that injects illumination context into the decoding process for physically grounded generation of albedo, metallic, and roughness maps, and (3) a geometry-guided inpainting module based on a large view synthesis model that…
Peer Reviews
Decision·ICLR 2026 Poster
- First of all, the work proposes the multi-branch generation design and the lighting-aware attention mechanism offers a novel way of disentangling albedo and metallic-roughness (MR) while integrating illumination context. - Quantitative results (FID, CMMD, LPIPS) and qualitative evaluations indicate that LumiTex achieves competitive or superior performance compared to existing methods, particularly in terms of texture quality and relighting fidelity. - The authors conducted a wide range of
- While the framework employs multi-branch generation and illumination context, similar ideas have already been explored in other recent works. For example, the idea of using lighting priors for material generation is not very novel. The originality of LumiTex comes into question because the combination of multi-view consistency and lighting-guided material attention don't significantly advance the state of the art in a groundbreaking way. - The method is computationally intensive and limited t
- The authors tackle an important problem. - The renderings of the shaded outputs look nice. - I appreciate the re-lighting results in the video.
The presentation is confusing, and I'm having trouble understanding several of the technical details: - The introduction mostly pitches the features but a coherent description of the core idea; e.g., how does the multi-view shaded image generator work is somewhat omitted – this makes it hard to read (e.g., first need to read the whole main section and even some of the results to understand which base models they were using) - Fig 3 is a pipeline but the description of the multi-view illuminati
To my knowledge, this is the first approach that is able to generate close-to-true PBR materials, without noticeable baked reflections or highlights. Given strong quantitative and qualitative evaluation, and the importance and complexity of the texturing task, I consider this work to be significant to the field. The main contributions and strengths of the paper are clearly demonstrated and ablated in section 4.5, namely: - Separate branch for shaded images prediction - Single-stage generation (
- Impact and novelty of geometry-guided inpainting module is limited, although this is not claimed as a main contribution of the paper.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComputer Graphics and Visualization Techniques · 3D Shape Modeling and Analysis · Generative Adversarial Networks and Image Synthesis