DeepHS-HDRVideo: Deep High Speed High Dynamic Range Video Reconstruction

TL;DR

This paper introduces a novel HDR video reconstruction method that uses video frame interpolation for better frame alignment, enabling high FPS HDR video generation with improved quality over traditional optical flow approaches.

Contribution

It is the first to utilize video frame interpolation for HDR video reconstruction, bypassing exposure matching and achieving high FPS HDR videos with superior alignment.

Findings

Achieves 2.4 PSNR improvement over optical flow methods

Enables high FPS HDR video generation through recursive interpolation

Demonstrates superior performance on standard HDR datasets

Abstract

Due to hardware constraints, standard off-the-shelf digital cameras suffers from low dynamic range (LDR) and low frame per second (FPS) outputs. Previous works in high dynamic range (HDR) video reconstruction uses sequence of alternating exposure LDR frames as input, and align the neighbouring frames using optical flow based networks. However, these methods often result in motion artifacts in challenging situations. This is because, the alternate exposure frames have to be exposure matched in order to apply alignment using optical flow. Hence, over-saturation and noise in the LDR frames results in inaccurate alignment. To this end, we propose to align the input LDR frames using a pre-trained video frame interpolation network. This results in better alignment of LDR frames, since we circumvent the error-prone exposure matching step, and directly generate intermediate missing frames from the same exposure inputs. Furthermore, it allows us to generate high FPS HDR videos by recursively interpolating the intermediate frames. Through this work, we propose to use video frame interpolation for HDR video reconstruction, and present the first method to generate high FPS HDR videos. Experimental results demonstrate the efficacy of the proposed framework against optical flow based alignment methods, with an absolute improvement of 2.4 PSNR value on standard HDR video datasets [1], [2] and further benchmark our method for high FPS HDR video generation.