Learning Relighting and Intrinsic Decomposition in Neural Radiance Fields

TL;DR

This paper presents a novel approach that combines relighting and intrinsic decomposition in neural radiance fields, enabling intrinsic component extraction in real scenes without ground truth, and demonstrating effectiveness in editing applications.

Contribution

It introduces a physically grounded method leveraging scene light variations for intrinsic decomposition in neural radiance fields, applicable to real scenes and image editing.

Findings

Effective on both synthetic and real datasets

Reduces reliance on ground truth and pre-trained models

Enables practical image editing applications

Abstract

The task of extracting intrinsic components, such as reflectance and shading, from neural radiance fields is of growing interest. However, current methods largely focus on synthetic scenes and isolated objects, overlooking the complexities of real scenes with backgrounds. To address this gap, our research introduces a method that combines relighting with intrinsic decomposition. By leveraging light variations in scenes to generate pseudo labels, our method provides guidance for intrinsic decomposition without requiring ground truth data. Our method, grounded in physical constraints, ensures robustness across diverse scene types and reduces the reliance on pre-trained models or hand-crafted priors. We validate our method on both synthetic and real-world datasets, achieving convincing results. Furthermore, the applicability of our method to image editing tasks demonstrates promising outcomes.