Deep Parametric Indoor Lighting Estimation

TL;DR

This paper introduces a deep learning method for estimating localized indoor lighting from a single image by representing lights as discrete 3D entities, improving accuracy and realism in 3D scene compositing.

Contribution

It proposes a novel parametric lighting representation and a differentiable layer for converting parameters to environment maps, enabling more accurate indoor lighting estimation.

Findings

Outperforms previous methods in accuracy

Enables realistic 3D object compositing

Uses a differentiable layer for parameter conversion

Abstract

We present a method to estimate lighting from a single image of an indoor scene. Previous work has used an environment map representation that does not account for the localized nature of indoor lighting. Instead, we represent lighting as a set of discrete 3D lights with geometric and photometric parameters. We train a deep neural network to regress these parameters from a single image, on a dataset of environment maps annotated with depth. We propose a differentiable layer to convert these parameters to an environment map to compute our loss; this bypasses the challenge of establishing correspondences between estimated and ground truth lights. We demonstrate, via quantitative and qualitative evaluations, that our representation and training scheme lead to more accurate results compared to previous work, while allowing for more realistic 3D object compositing with spatially-varying lighting.