Enhancing Multi-Exposure High Dynamic Range Imaging with Overlapped Codebook for Improved Representation Learning

TL;DR

This paper introduces an Overlapped codebook scheme to enhance multi-exposure HDR imaging by improving implicit representation learning, effectively handling saturation and motion discrepancies in LDR frames.

Contribution

The work proposes a novel Overlapped codebook for VQGAN to better model HDR representations and a new HDR network leveraging this for improved reconstruction.

Findings

Outperforms previous methods in qualitative assessments

Achieves higher quantitative accuracy in HDR reconstruction

Effectively handles saturated regions and motion discrepancies

Abstract

High dynamic range (HDR) imaging technique aims to create realistic HDR images from low dynamic range (LDR) inputs. Specifically, Multi-exposure HDR imaging uses multiple LDR frames taken from the same scene to improve reconstruction performance. However, there are often discrepancies in motion among the frames, and different exposure settings for each capture can lead to saturated regions. In this work, we first propose an Overlapped codebook (OLC) scheme, which can improve the capability of the VQGAN framework for learning implicit HDR representations by modeling the common exposure bracket process in the shared codebook structure. Further, we develop a new HDR network that utilizes HDR representations obtained from a pre-trained VQ network and OLC. This allows us to compensate for saturated regions and enhance overall visual quality. We have tested our approach extensively on various datasets and have demonstrated that it outperforms previous methods both qualitatively and quantitatively