Baking Relightable NeRF for Real-time Direct/Indirect Illumination Rendering
Euntae Choi, Vincent Carpentier, Seunghun Shin, Sungjoo Yoo

TL;DR
This paper introduces a real-time relighting method for NeRF-based rendering that efficiently handles direct and indirect illumination using CNN and hash grid-based renderers, enabling photo-realistic view synthesis under new lighting conditions.
Contribution
It presents a novel combination of CNN and hash grid-based renderers trained via distillation for fast, accurate relighting of NeRF models with minimal quality loss.
Findings
Achieves real-time relighting with negligible quality loss.
Effectively handles both direct and indirect illumination.
Enables photo-realistic view synthesis under unseen lighting.
Abstract
Relighting, which synthesizes a novel view under a given lighting condition (unseen in training time), is a must feature for immersive photo-realistic experience. However, real-time relighting is challenging due to high computation cost of the rendering equation which requires shape and material decomposition and visibility test to model shadow. Additionally, for indirect illumination, additional computation of rendering equation on each secondary surface point (where reflection occurs) is required rendering real-time relighting challenging. We propose a novel method that executes a CNN renderer to compute primary surface points and rendering parameters, required for direct illumination. We also present a lightweight hash grid-based renderer, for indirect illumination, which is recursively executed to perform the secondary ray tracing process. Both renderers are trained in a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComputer Graphics and Visualization Techniques · 3D Shape Modeling and Analysis · Advanced Vision and Imaging
