LM-GAN: A Photorealistic All-Weather Parametric Sky Model

TL;DR

LM-GAN is a novel HDR sky model that generates photorealistic, all-weather environment maps with diverse cloud formations, enabling realistic lighting in 3D rendering and image-based lighting applications.

Contribution

The paper introduces a generative model trained on sky appearance and a new sky-parameter fitting algorithm, improving realism and fidelity over existing methods.

Findings

Fitting algorithm surpasses existing approaches in accuracy

Generated environment maps match real HDR skies effectively

Model enables flexible, photorealistic sky generation for various weather conditions

Abstract

We present LM-GAN, an HDR sky model that generates photorealistic environment maps with weathered skies. Our sky model retains the flexibility of traditional parametric models and enables the reproduction of photorealistic all-weather skies with visual diversity in cloud formations. This is achieved with flexible and intuitive user controls for parameters, including sun position, sky color, and atmospheric turbidity. Our method is trained directly from inputs fitted to real HDR skies, learning both to preserve the input's illumination and correlate it to the real reference's atmospheric components in an end-to-end manner. Our main contributions are a generative model trained on both sky appearance and scene rendering losses, as well as a novel sky-parameter fitting algorithm. We demonstrate that our fitting algorithm surpasses existing approaches in both accuracy and sky fidelity, and also provide quantitative and qualitative analyses, demonstrating LM-GAN's ability to match parametric input to photorealistic all-weather skies. The generated HDR environment maps are ready to use in 3D rendering engines and can be applied to a wide range of image-based lighting applications.