Accidental Light Probes

TL;DR

This paper introduces a method to recover scene lighting from accidental light probes (ALPs) like shiny objects in images, enabling high-fidelity lighting estimation from common, everyday scenes.

Contribution

It presents a physically-based model and differentiable rendering approach to estimate lighting from accidental light probes in single images, a novel way to utilize incidental objects for illumination recovery.

Findings

Successfully estimates lighting from ALPs in single images.

Enables insertion of ALPs into scenes for lighting estimation.

Can recover lighting from existing images with accidental probes.

Abstract

Recovering lighting in a scene from a single image is a fundamental problem in computer vision. While a mirror ball light probe can capture omnidirectional lighting, light probes are generally unavailable in everyday images. In this work, we study recovering lighting from accidental light probes (ALPs) -- common, shiny objects like Coke cans, which often accidentally appear in daily scenes. We propose a physically-based approach to model ALPs and estimate lighting from their appearances in single images. The main idea is to model the appearance of ALPs by photogrammetrically principled shading and to invert this process via differentiable rendering to recover incidental illumination. We demonstrate that we can put an ALP into a scene to allow high-fidelity lighting estimation. Our model can also recover lighting for existing images that happen to contain an ALP.