DaLPSR: Leverage Degradation-Aligned Language Prompt for Real-World Image Super-Resolution

TL;DR

This paper introduces DaLPSR, a novel image super-resolution method that uses degradation-aligned language prompts and semantic priors to improve the fidelity and perceptual quality of restored images, especially in severely degraded cases.

Contribution

It proposes a degradation-aligned language prompt approach with an image-restoration prompt alignment decoder and semantic priors from large language models for enhanced super-resolution.

Findings

Achieves state-of-the-art perceptual quality on benchmark datasets.

Effectively discerns degradation levels to guide image restoration.

Outperforms existing methods in both qualitative and quantitative evaluations.

Abstract

Image super-resolution pursuits reconstructing high-fidelity high-resolution counterpart for low-resolution image. In recent years, diffusion-based models have garnered significant attention due to their capabilities with rich prior knowledge. The success of diffusion models based on general text prompts has validated the effectiveness of textual control in the field of text2image. However, given the severe degradation commonly presented in low-resolution images, coupled with the randomness characteristics of diffusion models, current models struggle to adequately discern semantic and degradation information within severely degraded images. This often leads to obstacles such as semantic loss, visual artifacts, and visual hallucinations, which pose substantial challenges for practical use. To address these challenges, this paper proposes to leverage degradation-aligned language prompt for accurate, fine-grained, and high-fidelity image restoration. Complementary priors including semantic content descriptions and degradation prompts are explored. Specifically, on one hand, image-restoration prompt alignment decoder is proposed to automatically discern the degradation degree of LR images, thereby generating beneficial degradation priors for image restoration. On the other hand, much richly tailored descriptions from pretrained multimodal large language model elicit high-level semantic priors closely aligned with human perception, ensuring fidelity control for image restoration. Comprehensive comparisons with state-of-the-art methods have been done on several popular synthetic and real-world benchmark datasets. The quantitative and qualitative analysis have demonstrated that the proposed method achieves a new state-of-the-art perceptual quality level. Related source codes and pre-trained parameters were public in https://github.com/puppy210/DaLPSR.