HDRFlow: Real-Time HDR Video Reconstruction with Large Motions

TL;DR

HDRFlow is a novel real-time HDR video reconstruction method that effectively handles large motions using a specialized flow estimator, HDR-domain alignment loss, and synthetic training data, outperforming previous approaches.

Contribution

The paper introduces HDRFlow, the first real-time HDR video reconstruction method capable of managing large motions efficiently with new loss functions and network designs.

Findings

Outperforms previous methods on standard benchmarks.

Processes 720p inputs at 25ms in real-time.

Effectively models large motions with minimal computational cost.

Abstract

Reconstructing High Dynamic Range (HDR) video from image sequences captured with alternating exposures is challenging, especially in the presence of large camera or object motion. Existing methods typically align low dynamic range sequences using optical flow or attention mechanism for deghosting. However, they often struggle to handle large complex motions and are computationally expensive. To address these challenges, we propose a robust and efficient flow estimator tailored for real-time HDR video reconstruction, named HDRFlow. HDRFlow has three novel designs: an HDR-domain alignment loss (HALoss), an efficient flow network with a multi-size large kernel (MLK), and a new HDR flow training scheme. The HALoss supervises our flow network to learn an HDR-oriented flow for accurate alignment in saturated and dark regions. The MLK can effectively model large motions at a negligible cost. In addition, we incorporate synthetic data, Sintel, into our training dataset, utilizing both its provided forward flow and backward flow generated by us to supervise our flow network, enhancing our performance in large motion regions. Extensive experiments demonstrate that our HDRFlow outperforms previous methods on standard benchmarks. To the best of our knowledge, HDRFlow is the first real-time HDR video reconstruction method for video sequences captured with alternating exposures, capable of processing 720p resolution inputs at 25ms.