Towards Physically-Based Sky-Modeling For Image Based Lighting

TL;DR

This paper introduces AllSky, a novel HDRI-based sky-model that enables photorealistic outdoor scene rendering with user-controlled sun and cloud positioning, surpassing existing DNN models in realism and flexibility.

Contribution

We propose AllSky, a physically-based, learnable sky-model from HDRI data that supports user control and achieves state-of-the-art performance in sky modeling.

Findings

AllSky outperforms existing DNN sky-models in realism and flexibility.

Current DNN models cannot match physically captured HDRI in scene re-lighting.

Existing models lack scalability and accuracy for downstream applications.

Abstract

Accurate environment maps are a key component for rendering photorealistic outdoor scenes with coherent illumination. They enable captivating visual arts, immersive virtual reality, and a wide range of engineering and scientific applications. Recent works have extended sky-models to be more comprehensive and inclusive of cloud formations but, as we demonstrate, existing methods fall short in faithfully recreating natural skies. Though in recent years the visual quality of DNN-generated High Dynamic Range Imagery (HDRI) has greatly improved, the environment maps generated by DNN sky-models do not re-light scenes with the same tones, shadows, and illumination as physically captured HDR imagery. In this work, we demonstrate progress in HDR literature to be tangential to sky-modelling as current works cannot support both photorealism and the 22 f-stops required for the Full Dynamic Range (FDR) of outdoor illumination. We achieve this by proposing AllSky, a flexible all-weather sky-model learned directly from physically captured HDRI which we leverage to study the input modalities, tonemapping, conditioning, and evaluation of sky-models. Per user-controlled positioning of the sun and cloud formations, AllSky expands on current functionality by allowing for intuitive user control over environment maps and achieves state-of-the-art sky-model performance. Through our proposed evaluation, we demonstrate existing DNN sky-models are not interchangeable with physically captured HDRI or parametric sky-models, with current limitations being prohibitive of scalability and accurate illumination in downstream applications