HDR Environment Map Estimation with Latent Diffusion Models

TL;DR

This paper introduces a novel HDR environment map estimation method using Latent Diffusion Models, addressing ERP distortions and seams with new padding and architecture, achieving high-quality results comparable to state-of-the-art methods.

Contribution

The paper proposes a new approach leveraging Latent Diffusion Models and a panoramically-adapted Diffusion Transformer to improve HDR environment map estimation from single images.

Findings

The ERP convolutional padding reduces border seams.

The PanoDiT architecture decreases ERP distortions.

The models achieve competitive lighting accuracy and image quality.

Abstract

We advance the field of HDR environment map estimation from a single-view image by establishing a novel approach leveraging the Latent Diffusion Model (LDM) to produce high-quality environment maps that can plausibly light mirror-reflective surfaces. A common issue when using the ERP representation, the format used by the vast majority of approaches, is distortions at the poles and a seam at the sides of the environment map. We remove the border seam artefact by proposing an ERP convolutional padding in the latent autoencoder. Additionally, we investigate whether adapting the diffusion network architecture to the ERP format can improve the quality and accuracy of the estimated environment map by proposing a panoramically-adapted Diffusion Transformer architecture. Our proposed PanoDiT network reduces ERP distortions and artefacts, but at the cost of image quality and plausibility. We evaluate with standard benchmarks to demonstrate that our models estimate high-quality environment maps that perform competitively with state-of-the-art approaches in both image quality and lighting accuracy.