Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Prior

TL;DR

This paper introduces a rotation-equivariant neural illumination prior using a novel neural field representation, improving natural environment map modeling for inverse rendering and environment map completion.

Contribution

It presents a new rotation-equivariant neural illumination model based on a variational auto-decoder and SIREN, capable of modeling complex natural environment maps.

Findings

Outperforms traditional spherical harmonic representations

Enables environment map completion from partial data

Demonstrates effectiveness in inverse rendering tasks

Abstract

Inverse rendering is an ill-posed problem. Previous work has sought to resolve this by focussing on priors for object or scene shape or appearance. In this work, we instead focus on a prior for natural illuminations. Current methods rely on spherical harmonic lighting or other generic representations and, at best, a simplistic prior on the parameters. We propose a conditional neural field representation based on a variational auto-decoder with a SIREN network and, extending Vector Neurons, build equivariance directly into the network. Using this, we develop a rotation-equivariant, high dynamic range (HDR) neural illumination model that is compact and able to express complex, high-frequency features of natural environment maps. Training our model on a curated dataset of 1.6K HDR environment maps of natural scenes, we compare it against traditional representations, demonstrate its applicability for an inverse rendering task and show environment map completion from partial observations. A PyTorch implementation, our dataset and trained models can be found at jadgardner.github.io/RENI.