Are Conditional Latent Diffusion Models Effective for Image Restoration?

TL;DR

This paper critically evaluates the effectiveness of conditional latent diffusion models for image restoration, revealing their limitations in perceptual quality and suggesting the need for rethinking their application in this domain.

Contribution

The study provides a comprehensive comparison between CLDMs and traditional models for IR, highlighting their shortcomings and analyzing design factors affecting performance.

Findings

CLDMs face high distortion in IR tasks

Traditional methods outperform CLDMs with minimal degradation

Design elements significantly impact CLDM restoration performance

Abstract

Recent advancements in image restoration increasingly employ conditional latent diffusion models (CLDMs). While these models have demonstrated notable performance improvements in recent years, this work questions their suitability for IR tasks. CLDMs excel in capturing high-level semantic correlations, making them effective for tasks like text-to-image generation with spatial conditioning. However, in IR, where the goal is to enhance image perceptual quality, these models face difficulty of modeling the relationship between degraded images and ground truth images using a low-level representation. To support our claims, we compare state-of-the-art CLDMs with traditional image restoration models through extensive experiments. Results reveal that despite the scaling advantages of CLDMs, they suffer from high distortion and semantic deviation, especially in cases with minimal degradation, where traditional methods outperform them. Additionally, we perform empirical studies to examine the impact of various CLDM design elements on their restoration performance. We hope this finding inspires a reexamination of current CLDM-based IR solutions, opening up more opportunities in this field.