Learning to Estimate Indoor Lighting from 3D Objects

TL;DR

This paper introduces a deep learning approach that accurately predicts indoor lighting from a single image of a known object by learning a compact lighting representation and using a large HDR environment map dataset.

Contribution

The paper presents a novel autoencoder for modeling indoor lighting and a CNN for predicting lighting from images, trained on a new extensive HDR environment map dataset.

Findings

The method produces more realistic lighting predictions than previous approaches.

It can estimate plausible indoor lighting from diffuse objects.

The approach demonstrates high accuracy on a large database of environment maps.

Abstract

In this work, we propose a step towards a more accurate prediction of the environment light given a single picture of a known object. To achieve this, we developed a deep learning method that is able to encode the latent space of indoor lighting using few parameters and that is trained on a database of environment maps. This latent space is then used to generate predictions of the light that are both more realistic and accurate than previous methods. To achieve this, our first contribution is a deep autoencoder which is capable of learning the feature space that compactly models lighting. Our second contribution is a convolutional neural network that predicts the light from a single image of a known object. To train these networks, our third contribution is a novel dataset that contains 21,000 HDR indoor environment maps. The results indicate that the predictor can generate plausible lighting estimations even from diffuse objects.