Wavelet-Domain Masked Image Modeling for Color-Consistent HDR Video Reconstruction

TL;DR

This paper introduces WMNet, a wavelet-domain masked image modeling approach for HDR video reconstruction that enhances color accuracy and temporal consistency using novel modules and a new benchmark dataset.

Contribution

The paper presents WMNet, a novel HDR video reconstruction network employing wavelet domain masked image modeling and new modules for improved color and temporal fidelity.

Findings

Achieves state-of-the-art performance in HDR video reconstruction

Significantly improves color fidelity and temporal coherence

Introduces HDRTV4K-Scene benchmark dataset

Abstract

High Dynamic Range (HDR) video reconstruction aims to recover fine brightness, color, and details from Low Dynamic Range (LDR) videos. However, existing methods often suffer from color inaccuracies and temporal inconsistencies. To address these challenges, we propose WMNet, a novel HDR video reconstruction network that leverages Wavelet domain Masked Image Modeling (W-MIM). WMNet adopts a two-phase training strategy: In Phase I, W-MIM performs self-reconstruction pre-training by selectively masking color and detail information in the wavelet domain, enabling the network to develop robust color restoration capabilities. A curriculum learning scheme further refines the reconstruction process. Phase II fine-tunes the model using the pre-trained weights to improve the final reconstruction quality. To improve temporal consistency, we introduce the Temporal Mixture of Experts (T-MoE) module and the Dynamic Memory Module (DMM). T-MoE adaptively fuses adjacent frames to reduce flickering artifacts, while DMM captures long-range dependencies, ensuring smooth motion and preservation of fine details. Additionally, since existing HDR video datasets lack scene-based segmentation, we reorganize HDRTV4K into HDRTV4K-Scene, establishing a new benchmark for HDR video reconstruction. Extensive experiments demonstrate that WMNet achieves state-of-the-art performance across multiple evaluation metrics, significantly improving color fidelity, temporal coherence, and perceptual quality. The code is available at: https://github.com/eezkni/WMNet