ICM-SR: Image-Conditioned Manifold Regularization for Image Super-Resolution

TL;DR

This paper introduces ICM-SR, a novel image-conditioned manifold regularization method for image super-resolution that leverages structural cues like colormap and edges to improve perceptual quality.

Contribution

It proposes a stable, task-aligned regularization technique using sparse structural information, addressing limitations of previous manifold-based super-resolution methods.

Findings

ICM-SR significantly improves perceptual quality in super-resolution tasks.

The method effectively stabilizes regularization by conditioning on structural cues.

Experiments demonstrate superior performance over existing approaches.

Abstract

Real world image super-resolution (Real-ISR) often leverages the powerful generative priors of text-to-image diffusion models by regularizing the output to lie on their learned manifold. However, existing methods often overlook the importance of the regularizing manifold, typically defaulting to a text-conditioned manifold. This approach suffers from two key limitations. Conceptually, it is misaligned with the Real-ISR task, which is to generate high quality (HQ) images directly tied to the low quality (LQ) images. Practically, the teacher model often reconstructs images with color distortions and blurred edges, indicating a flawed generative prior for this task. To correct these flaws and ensure conceptual alignment, a more suitable manifold must incorporate information from the images. While the most straightforward approach is to condition directly on the raw input images, their high information densities make the regularization process numerically unstable. To resolve this, we propose image-conditioned manifold regularization (ICM), a method that regularizes the output towards a manifold conditioned on the sparse yet essential structural information: a combination of colormap and Canny edges. ICM provides a task-aligned and stable regularization signal, thereby avoiding the instability of dense-conditioning and enhancing the final super-resolution quality. Our experiments confirm that the proposed regularization significantly enhances super-resolution performance, particularly in perceptual quality, demonstrating its effectiveness for real-world applications. We will release the source code of our work for reproducibility.