Efficient Image Restoration through Low-Rank Adaptation and Stable Diffusion XL

TL;DR

This paper introduces SUPIR, a novel image restoration model that combines low-rank adaptive modules with Stable Diffusion XL, achieving superior quality and efficiency in restoring high-fidelity images.

Contribution

The paper presents a new method integrating LoRA modules with SDXL for enhanced image restoration, trained on a large real-world dataset, and validated on standard benchmarks.

Findings

Higher PSNR scores on benchmarks

Lower LPIPS indicating better perceptual quality

Higher SSIM demonstrating improved structural similarity

Abstract

In this study, we propose an enhanced image restoration model, SUPIR, based on the integration of two low-rank adaptive (LoRA) modules with the Stable Diffusion XL (SDXL) framework. Our method leverages the advantages of LoRA to fine-tune SDXL models, thereby significantly improving image restoration quality and efficiency. We collect 2600 high-quality real-world images, each with detailed descriptive text, for training the model. The proposed method is evaluated on standard benchmarks and achieves excellent performance, demonstrated by higher peak signal-to-noise ratio (PSNR), lower learned perceptual image patch similarity (LPIPS), and higher structural similarity index measurement (SSIM) scores. These results underscore the effectiveness of combining LoRA with SDXL for advanced image restoration tasks, highlighting the potential of our approach in generating high-fidelity restored images.