Eclipse: Disambiguating Illumination and Materials using Unintended Shadows

TL;DR

This paper introduces a novel inverse rendering method that leverages unintended shadows cast on diffuse objects to accurately recover their materials and surrounding illumination, overcoming traditional ambiguities.

Contribution

It demonstrates how unintended shadows can be exploited as signals to improve the disambiguation of materials and lighting in inverse rendering, using differentiable Monte Carlo ray tracing.

Findings

Successfully recovers spatially-varying materials and illumination from images.

Utilizes shadows cast by unseen occluders to resolve ambiguities.

Enhances inverse rendering accuracy for diffuse objects.

Abstract

Decomposing an object's appearance into representations of its materials and the surrounding illumination is difficult, even when the object's 3D shape is known beforehand. This problem is especially challenging for diffuse objects: it is ill-conditioned because diffuse materials severely blur incoming light, and it is ill-posed because diffuse materials under high-frequency lighting can be indistinguishable from shiny materials under low-frequency lighting. We show that it is possible to recover precise materials and illumination -- even from diffuse objects -- by exploiting unintended shadows, like the ones cast onto an object by the photographer who moves around it. These shadows are a nuisance in most previous inverse rendering pipelines, but here we exploit them as signals that improve conditioning and help resolve material-lighting ambiguities. We present a method based on differentiable Monte Carlo ray tracing that uses images of an object to jointly recover its spatially-varying materials, the surrounding illumination environment, and the shapes of the unseen light occluders who inadvertently cast shadows upon it.