Restoring Real-World Images with an Internal Detail Enhancement Diffusion Model

TL;DR

This paper introduces an internal detail-preserving diffusion model that leverages a pre-trained Stable Diffusion prior and a novel IIDE technique to restore real-world degraded images with high fidelity and object-level control, outperforming existing methods.

Contribution

The paper presents a novel internal detail enhancement diffusion approach that preserves structural details during restoration and enables text-guided, object-level colorization control.

Findings

Outperforms state-of-the-art models in qualitative and quantitative evaluations.

Supports text-guided restoration for object-level colorization.

Effectively preserves image details during complex degradation removal.

Abstract

Restoring real-world degraded images, such as old photographs or low-resolution images, presents a significant challenge due to the complex, mixed degradations they exhibit, such as scratches, color fading, and noise. Recent data-driven approaches have struggled with two main challenges: achieving high-fidelity restoration and providing object-level control over colorization. While diffusion models have shown promise in generating high-quality images with specific controls, they often fail to fully preserve image details during restoration. In this work, we propose an internal detail-preserving diffusion model for high-fidelity restoration of real-world degraded images. Our method utilizes a pre-trained Stable Diffusion model as a generative prior, eliminating the need to train a model from scratch. Central to our approach is the Internal Image Detail Enhancement (IIDE) technique, which directs the diffusion model to preserve essential structural and textural information while mitigating degradation effects. The process starts by mapping the input image into a latent space, where we inject the diffusion denoising process with degradation operations that simulate the effects of various degradation factors. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art models in both qualitative assessments and perceptual quantitative evaluations. Additionally, our approach supports text-guided restoration, enabling object-level colorization control that mimics the expertise of professional photo editing.