Online Overexposed Pixels Hallucination in Videos with Adaptive Reference Frame Selection
Yazhou Xing, Amrita Mazumdar, Anjul Patney, Chao Liu, Hongxu Yin,, Qifeng Chen, Jan Kautz, Iuri Frosio

TL;DR
This paper introduces a learning-based, transformer-driven video HDR hallucination method that adaptively selects reference frames to reduce overexposure artifacts without complex HDR capture techniques.
Contribution
It presents a novel adaptive reference frame selection mechanism using reinforcement learning to improve HDR detail inference in videos with overexposed pixels.
Findings
Achieves state-of-the-art HDR hallucination quality
Effectively leverages temporal instabilities for better reconstruction
Operates causally without complex acquisition hardware
Abstract
Low dynamic range (LDR) cameras cannot deal with wide dynamic range inputs, frequently leading to local overexposure issues. We present a learning-based system to reduce these artifacts without resorting to complex acquisition mechanisms like alternating exposures or costly processing that are typical of high dynamic range (HDR) imaging. We propose a transformer-based deep neural network (DNN) to infer the missing HDR details. In an ablation study, we show the importance of using a multiscale DNN and train it with the proper cost function to achieve state-of-the-art quality. To aid the reconstruction of the overexposed areas, our DNN takes a reference frame from the past as an additional input. This leverages the commonly occurring temporal instabilities of autoexposure to our advantage: since well-exposed details in the current frame may be overexposed in the future, we use…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image Processing Techniques · Image and Signal Denoising Methods · Image Enhancement Techniques
