Towards Real-World Burst Image Super-Resolution: Benchmark and Method

TL;DR

This paper introduces a large-scale real-world burst super-resolution dataset and a novel federated network that employs homography alignment and transformer modules to improve image reconstruction quality in realistic scenarios.

Contribution

It presents a new dataset for real-world burst super-resolution and a federated network with homography alignment and transformer-based decoding, advancing the state-of-the-art.

Findings

FBAnet outperforms existing burst SR methods.

The dataset enables realistic evaluation of super-resolution models.

The method produces visually pleasing high-resolution images.

Abstract

Despite substantial advances, single-image super-resolution (SISR) is always in a dilemma to reconstruct high-quality images with limited information from one input image, especially in realistic scenarios. In this paper, we establish a large-scale real-world burst super-resolution dataset, i.e., RealBSR, to explore the faithful reconstruction of image details from multiple frames. Furthermore, we introduce a Federated Burst Affinity network (FBAnet) to investigate non-trivial pixel-wise displacements among images under real-world image degradation. Specifically, rather than using pixel-wise alignment, our FBAnet employs a simple homography alignment from a structural geometry aspect and a Federated Affinity Fusion (FAF) strategy to aggregate the complementary information among frames. Those fused informative representations are fed to a Transformer-based module of burst representation decoding. Besides, we have conducted extensive experiments on two versions of our datasets, i.e., RealBSR-RAW and RealBSR-RGB. Experimental results demonstrate that our FBAnet outperforms existing state-of-the-art burst SR methods and also achieves visually-pleasant SR image predictions with model details. Our dataset, codes, and models are publicly available at https://github.com/yjsunnn/FBANet.