# Deep Sky Modeling for Single Image Outdoor Lighting Estimation

## TL;DR

This paper introduces a data-driven sky model for estimating outdoor lighting from a single image, enabling realistic virtual object insertion without relying on analytical models.

## Contribution

It presents a holistic, end-to-end approach combining diverse datasets to learn a sky model that estimates HDR environment maps directly from images.

## Key findings

- Outperforms previous state-of-the-art in outdoor lighting estimation
- Can generate plausible HDR environment maps from single images
- Enables visually convincing virtual object insertions

## Abstract

We propose a data-driven learned sky model, which we use for outdoor lighting estimation from a single image. As no large-scale dataset of images and their corresponding ground truth illumination is readily available, we use complementary datasets to train our approach, combining the vast diversity of illumination conditions of SUN360 with the radiometrically calibrated and physically accurate Laval HDR sky database. Our key contribution is to provide a holistic view of both lighting modeling and estimation, solving both problems end-to-end. From a test image, our method can directly estimate an HDR environment map of the lighting without relying on analytical lighting models. We demonstrate the versatility and expressivity of our learned sky model and show that it can be used to recover plausible illumination, leading to visually pleasant virtual object insertions. To further evaluate our method, we capture a dataset of HDR 360{\deg} panoramas and show through extensive validation that we significantly outperform previous state-of-the-art.