Ghost-free High Dynamic Range Imaging with Context-aware Transformer

TL;DR

This paper introduces a novel dual-branch Vision Transformer architecture for ghost-free HDR imaging, effectively capturing global and local dependencies to reduce artifacts and distortions caused by motion and saturation.

Contribution

The paper proposes a dual-branch CA-ViT architecture combining global and local context modeling for improved HDR deghosting, which outperforms existing CNN-based methods.

Findings

Outperforms state-of-the-art methods quantitatively and qualitatively

Reduces computational costs significantly

Effectively handles large motion and saturation artifacts

Abstract

High dynamic range (HDR) deghosting algorithms aim to generate ghost-free HDR images with realistic details. Restricted by the locality of the receptive field, existing CNN-based methods are typically prone to producing ghosting artifacts and intensity distortions in the presence of large motion and severe saturation. In this paper, we propose a novel Context-Aware Vision Transformer (CA-ViT) for ghost-free high dynamic range imaging. The CA-ViT is designed as a dual-branch architecture, which can jointly capture both global and local dependencies. Specifically, the global branch employs a window-based Transformer encoder to model long-range object movements and intensity variations to solve ghosting. For the local branch, we design a local context extractor (LCE) to capture short-range image features and use the channel attention mechanism to select informative local details across the extracted features to complement the global branch. By incorporating the CA-ViT as basic components, we further build the HDR-Transformer, a hierarchical network to reconstruct high-quality ghost-free HDR images. Extensive experiments on three benchmark datasets show that our approach outperforms state-of-the-art methods qualitatively and quantitatively with considerably reduced computational budgets. Codes are available at https://github.com/megvii-research/HDR-Transformer