BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment

TL;DR

This paper introduces BSRT, a novel transformer-based architecture with flow-guided deformable alignment for burst super-resolution, achieving state-of-the-art results and winning the NTIRE2022 challenge.

Contribution

It presents a new Burst Super-Resolution Transformer (BSRT) that combines Swin Transformer blocks with flow-guided deformable convolutions for improved multi-frame image reconstruction.

Findings

Achieves new state-of-the-art performance on synthetic and real-world datasets.

Wins the NTIRE2022 Burst Super-Resolution Challenge.

Effectively handles misalignment and extracts texture information across frames.

Abstract

This work addresses the Burst Super-Resolution (BurstSR) task using a new architecture, which requires restoring a high-quality image from a sequence of noisy, misaligned, and low-resolution RAW bursts. To overcome the challenges in BurstSR, we propose a Burst Super-Resolution Transformer (BSRT), which can significantly improve the capability of extracting inter-frame information and reconstruction. To achieve this goal, we propose a Pyramid Flow-Guided Deformable Convolution Network (Pyramid FG-DCN) and incorporate Swin Transformer Blocks and Groups as our main backbone. More specifically, we combine optical flows and deformable convolutions, hence our BSRT can handle misalignment and aggregate the potential texture information in multi-frames more efficiently. In addition, our Transformer-based structure can capture long-range dependency to further improve the performance. The evaluation on both synthetic and real-world tracks demonstrates that our approach achieves a new state-of-the-art in BurstSR task. Further, our BSRT wins the championship in the NTIRE2022 Burst Super-Resolution Challenge.