Ultra-High-Definition Reference-Based Landmark Image Super-Resolution with Generative Diffusion Prior

TL;DR

This paper introduces TriFlowSR, a diffusion-based super-resolution framework that effectively matches low-resolution images with high-resolution references, supported by a new UHD landmark dataset, achieving superior detail restoration in real-world scenarios.

Contribution

The paper presents the first diffusion-based RefSR pipeline for UHD landmark images and introduces Landmark-4K, a high-quality dataset for this task.

Findings

TriFlowSR outperforms previous methods in utilizing reference images.

Effective pattern matching improves super-resolution quality.

First UHD landmark RefSR dataset with real-world degradation.

Abstract

Reference-based Image Super-Resolution (RefSR) aims to restore a low-resolution (LR) image by utilizing the semantic and texture information from an additional reference high-resolution (reference HR) image. Existing diffusion-based RefSR methods are typically built upon ControlNet, which struggles to effectively align the information between the LR image and the reference HR image. Moreover, current RefSR datasets suffer from limited resolution and poor image quality, resulting in the reference images lacking sufficient fine-grained details to support high-quality restoration. To overcome the limitations above, we propose TriFlowSR, a novel framework that explicitly achieves pattern matching between the LR image and the reference HR image. Meanwhile, we introduce Landmark-4K, the first RefSR dataset for Ultra-High-Definition (UHD) landmark scenarios. Considering the UHD scenarios with real-world degradation, in TriFlowSR, we design a Reference Matching Strategy to effectively match the LR image with the reference HR image. Experimental results show that our approach can better utilize the semantic and texture information of the reference HR image compared to previous methods. To the best of our knowledge, we propose the first diffusion-based RefSR pipeline for ultra-high definition landmark scenarios under real-world degradation. Our code and model will be available at https://github.com/nkicsl/TriFlowSR.