DACESR: Degradation-Aware Conditional Embedding for Real-World Image Super-Resolution

TL;DR

This paper introduces DACESR, a degradation-aware conditional embedding method that enhances real-world image super-resolution by employing a degradation selection strategy and a conditional feature modulator to improve recognition and restoration quality.

Contribution

It proposes a novel degradation-aware embedding framework with a Real Embedding Extractor and Conditional Feature Modulator for improved super-resolution of degraded images.

Findings

REE significantly improves recognition on degraded images.

The method balances fidelity and perceptual quality effectively.

Extensive experiments validate the approach's superiority.

Abstract

Multimodal large models have shown excellent ability in addressing image super-resolution in real-world scenarios by leveraging language class as condition information, yet their abilities in degraded images remain limited. In this paper, we first revisit the capabilities of the Recognize Anything Model (RAM) for degraded images by calculating text similarity. We find that directly using contrastive learning to fine-tune RAM in the degraded space is difficult to achieve acceptable results. To address this issue, we employ a degradation selection strategy to propose a Real Embedding Extractor (REE), which achieves significant recognition performance gain on degraded image content through contrastive learning. Furthermore, we use a Conditional Feature Modulator (CFM) to incorporate the high-level information of REE for a powerful Mamba-based network, which can leverage effective pixel information to restore image textures and produce visually pleasing results. Extensive experiments demonstrate that the REE can effectively help image super-resolution networks balance fidelity and perceptual quality, highlighting the great potential of Mamba in real-world applications. The source code of this work will be made publicly available at: https://github.com/nathan66666/DACESR.git