Ghost-free High Dynamic Range Imaging via Hybrid CNN-Transformer and Structure Tensor

TL;DR

This paper introduces a hybrid CNN-Transformer model with structure tensor integration for ghost-free HDR imaging, effectively removing artifacts caused by moving objects and capable of handling arbitrary input images.

Contribution

The novel hybrid CNN-Transformer architecture with structure tensor guidance improves HDR deghosting performance and flexibility over existing methods.

Findings

Outperforms state-of-the-art HDR deghosting models in qualitative and quantitative tests.

Capable of processing an arbitrary number of input LDR images.

Effectively removes ghosting artifacts caused by moving objects.

Abstract

Eliminating ghosting artifacts due to moving objects is a challenging problem in high dynamic range (HDR) imaging. In this letter, we present a hybrid model consisting of a convolutional encoder and a Transformer decoder to generate ghost-free HDR images. In the encoder, a context aggregation network and non-local attention block are adopted to optimize multi-scale features and capture both global and local dependencies of multiple low dynamic range (LDR) images. The decoder based on Swin Transformer is utilized to improve the reconstruction capability of the proposed model. Motivated by the phenomenal difference between the presence and absence of artifacts under the field of structure tensor (ST), we integrate the ST information of LDR images as auxiliary inputs of the network and use ST loss to further constrain artifacts. Different from previous approaches, our network is capable of processing an arbitrary number of input LDR images. Qualitative and quantitative experiments demonstrate the effectiveness of the proposed method by comparing it with existing state-of-the-art HDR deghosting models. Codes are available at https://github.com/pandayuanyu/HSTHdr.