SILT: Self-supervised Lighting Transfer Using Implicit Image Decomposition

TL;DR

SILT is a self-supervised method that transfers lighting styles between scenes by mapping images to a unified domain and remapping them using style references, outperforming supervised methods without needing lighting labels.

Contribution

It introduces a novel self-supervised lighting transfer approach using implicit image decomposition and a two-branch network to transfer lighting styles without supervision.

Findings

Outperforms supervised relighting methods on two datasets

Does not require lighting supervision for training

Effective in transferring lighting styles across diverse scenes

Abstract

We present SILT, a Self-supervised Implicit Lighting Transfer method. Unlike previous research on scene relighting, we do not seek to apply arbitrary new lighting configurations to a given scene. Instead, we wish to transfer the lighting style from a database of other scenes, to provide a uniform lighting style regardless of the input. The solution operates as a two-branch network that first aims to map input images of any arbitrary lighting style to a unified domain, with extra guidance achieved through implicit image decomposition. We then remap this unified input domain using a discriminator that is presented with the generated outputs and the style reference, i.e. images of the desired illumination conditions. Our method is shown to outperform supervised relighting solutions across two different datasets without requiring lighting supervision.