Casual Indoor HDR Radiance Capture from Omnidirectional Images

TL;DR

This paper introduces PanoHDR-NeRF, a method for casually capturing and representing full HDR radiance in indoor scenes using simple omnidirectional video and neural rendering, enabling realistic lighting and scene synthesis.

Contribution

It presents a novel pipeline combining LDR to HDR conversion and neural radiance fields for easy indoor HDR scene capture without complex setups.

Findings

Accurately predicts HDR radiance from unseen scene points.

Enables realistic lighting effects for synthetic object insertion.

Works with casual, off-the-shelf camera footage.

Abstract

We present PanoHDR-NeRF, a neural representation of the full HDR radiance field of an indoor scene, and a pipeline to capture it casually, without elaborate setups or complex capture protocols. First, a user captures a low dynamic range (LDR) omnidirectional video of the scene by freely waving an off-the-shelf camera around the scene. Then, an LDR2HDR network uplifts the captured LDR frames to HDR, which are used to train a tailored NeRF++ model. The resulting PanoHDR-NeRF can render full HDR images from any location of the scene. Through experiments on a novel test dataset of real scenes with the ground truth HDR radiance captured at locations not seen during training, we show that PanoHDR-NeRF predicts plausible HDR radiance from any scene point. We also show that the predicted radiance can synthesize correct lighting effects, enabling the augmentation of indoor scenes with synthetic objects that are lit correctly. Datasets and code are available at https://lvsn.github.io/PanoHDR-NeRF/.