Illuminant Spectra-based Source Separation Using Flash Photography

TL;DR

This paper introduces a novel method leveraging flash/no-flash image pairs to automatically separate multiple scene illuminants based on their spectral differences, enabling improved lighting editing and analysis.

Contribution

It presents a physics-based relationship between color variations and spectral properties, enabling automatic illuminant separation without manual input or scene calibration.

Findings

Outperforms state-of-the-art techniques in illuminant separation

Enables applications like white balancing and lighting editing

Works across a wide range of images

Abstract

Real-world lighting often consists of multiple illuminants with different spectra. Separating and manipulating these illuminants in post-process is a challenging problem that requires either significant manual input or calibrated scene geometry and lighting. In this work, we leverage a flash/no-flash image pair to analyze and edit scene illuminants based on their spectral differences. We derive a novel physics-based relationship between color variations in the observed flash/no-flash intensities and the spectra and surface shading corresponding to individual scene illuminants. Our technique uses this constraint to automatically separate an image into constituent images lit by each illuminant. This separation can be used to support applications like white balancing, lighting editing, and RGB photometric stereo, where we demonstrate results that outperform state-of-the-art techniques on a wide range of images.