RRM: Relightable assets using Radiance guided Material extraction

TL;DR

RRM is a novel method that accurately extracts materials, geometry, and lighting from scenes with reflective objects, enabling high-quality relighting and view synthesis by combining physically-aware radiance fields with a Laplacian Pyramid-based environment model.

Contribution

It introduces a physically-aware radiance field and an expressive environment lighting structure to handle glossy scenes, surpassing previous methods in parameter retrieval and relighting quality.

Findings

Outperforms state-of-the-art in parameter retrieval.

Enables high-fidelity relighting of glossy scenes.

Achieves superior novel view synthesis results.

Abstract

Synthesizing NeRFs under arbitrary lighting has become a seminal problem in the last few years. Recent efforts tackle the problem via the extraction of physically-based parameters that can then be rendered under arbitrary lighting, but they are limited in the range of scenes they can handle, usually mishandling glossy scenes. We propose RRM, a method that can extract the materials, geometry, and environment lighting of a scene even in the presence of highly reflective objects. Our method consists of a physically-aware radiance field representation that informs physically-based parameters, and an expressive environment light structure based on a Laplacian Pyramid. We demonstrate that our contributions outperform the state-of-the-art on parameter retrieval tasks, leading to high-fidelity relighting and novel view synthesis on surfacic scenes.