A Maximum A Posteriori Estimation Framework for Robust High Dynamic Range Video Synthesis

TL;DR

This paper introduces a MAP-based framework for robust HDR video synthesis that bypasses the need for exact correspondence estimation, using novel optimization techniques for foreground and background extraction, resulting in higher quality videos.

Contribution

It proposes a new statistical approach for HDR video synthesis that avoids explicit correspondence estimation, employing rank minimization and multiscale kernel regression for improved results.

Findings

Outperforms state-of-the-art HDR video synthesis methods in quality.

Achieves better complexity-performance trade-off.

Demonstrates robustness on real and synthetic datasets.

Abstract

High dynamic range (HDR) image synthesis from multiple low dynamic range (LDR) exposures continues to be actively researched. The extension to HDR video synthesis is a topic of significant current interest due to potential cost benefits. For HDR video, a stiff practical challenge presents itself in the form of accurate correspondence estimation of objects between video frames. In particular, loss of data resulting from poor exposures and varying intensity make conventional optical flow methods highly inaccurate. We avoid exact correspondence estimation by proposing a statistical approach via maximum a posterior (MAP) estimation, and under appropriate statistical assumptions and choice of priors and models, we reduce it to an optimization problem of solving for the foreground and background of the target frame. We obtain the background through rank minimization and estimate the foreground via a novel multiscale adaptive kernel regression technique, which implicitly captures local structure and temporal motion by solving an unconstrained optimization problem. Extensive experimental results on both real and synthetic datasets demonstrate that our algorithm is more capable of delivering high-quality HDR videos than current state-of-the-art methods, under both subjective and objective assessments. Furthermore, a thorough complexity analysis reveals that our algorithm achieves better complexity-performance trade-off than conventional methods.