Towards Physically-Based Sky-Modeling

TL;DR

This paper introduces AllSky, a sky-model that learns from physically captured HDR images to produce environment maps with accurate lighting, shadows, and dynamic range, improving photorealistic outdoor scene rendering.

Contribution

We propose a novel all-weather sky-model that directly learns from HDR imagery, enabling realistic emulation of sky conditions with user-controlled sun and cloud positioning.

Findings

AllSky accurately reproduces physically captured environment maps.

The model maintains high Extended Dynamic Range (EDR) for outdoor scenes.

It outperforms existing sky-models in relighting scenes with realistic tones and shadows.

Abstract

Accurate environment maps are a key component in rendering photorealistic outdoor scenes with coherent illumination. They enable captivating visual arts, immersive virtual reality and a wide range of engineering and scientific applications. Recent works have extended sky-models to be more comprehensive and inclusive of cloud formations but existing approaches fall short in faithfully recreating key-characteristics in physically captured HDRI. As we demonstrate, environment maps produced by sky-models do not relight scenes with the same tones, shadows, and illumination coherence as physically captured HDR imagery. Though the visual quality of DNN-generated LDR and HDR imagery has greatly progressed in recent years, we demonstrate this progress to be tangential to sky-modelling. Due to the Extended Dynamic Range (EDR) of 14EV required for outdoor environment maps inclusive of the sun, sky-modelling extends beyond the conventional paradigm of High Dynamic Range Imagery (HDRI). In this work, we propose an all-weather sky-model, learning weathered-skies directly from physically captured HDR imagery. Per user-controlled positioning of the sun and cloud formations, our model (AllSky) allows for emulation of physically captured environment maps with improved retention of the Extended Dynamic Range (EDR) of the sky.