IFT: Image Fusion Transformer for Ghost-free High Dynamic Range Imaging

TL;DR

The paper introduces IFT, a novel transformer-based approach for ghost-free HDR imaging from misaligned LDR images, effectively modeling long-range dependencies and reducing artifacts in dynamic scenes.

Contribution

It proposes a new image fusion transformer with a fast global patch searching module and a self-cross fusion module, improving alignment and artifact reduction in HDR imaging.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Effectively models long-range dependencies in dynamic scenes.

Reduces ghosting artifacts compared to existing methods.

Abstract

Multi-frame high dynamic range (HDR) imaging aims to reconstruct ghost-free images with photo-realistic details from content-complementary but spatially misaligned low dynamic range (LDR) images. Existing HDR algorithms are prone to producing ghosting artifacts as their methods fail to capture long-range dependencies between LDR frames with large motion in dynamic scenes. To address this issue, we propose a novel image fusion transformer, referred to as IFT, which presents a fast global patch searching (FGPS) module followed by a self-cross fusion module (SCF) for ghost-free HDR imaging. The FGPS searches the patches from supporting frames that have the closest dependency to each patch of the reference frame for long-range dependency modeling, while the SCF conducts intra-frame and inter-frame feature fusion on the patches obtained by the FGPS with linear complexity to input resolution. By matching similar patches between frames, objects with large motion ranges in dynamic scenes can be aligned, which can effectively alleviate the generation of artifacts. In addition, the proposed FGPS and SCF can be integrated into various deep HDR methods as efficient plug-in modules. Extensive experiments on multiple benchmarks show that our method achieves state-of-the-art performance both quantitatively and qualitatively.