DriftRec: Adapting diffusion models to blind JPEG restoration

TL;DR

DriftRec introduces a diffusion model-based method for blind JPEG restoration that adapts the forward SDE to better recover high-quality images from highly compressed JPEGs without prior knowledge of the corruption process.

Contribution

The paper proposes a novel modification to diffusion models' forward SDE, enabling effective blind JPEG restoration with fewer sampling steps and improved image fidelity compared to existing methods.

Findings

Outperforms $L_2$ regression baseline and state-of-the-art techniques

Recovers image distribution more faithfully, reducing blurriness

Generalizes well to real-world scenarios like unaligned double JPEG compression

Abstract

In this work, we utilize the high-fidelity generation abilities of diffusion models to solve blind JPEG restoration at high compression levels. We propose an elegant modification of the forward stochastic differential equation of diffusion models to adapt them to this restoration task and name our method DriftRec. Comparing DriftRec against an $L_2$ regression baseline with the same network architecture and state-of-the-art techniques for JPEG restoration, we show that our approach can escape the tendency of other methods to generate blurry images, and recovers the distribution of clean images significantly more faithfully. For this, only a dataset of clean/corrupted image pairs and no knowledge about the corruption operation is required, enabling wider applicability to other restoration tasks. In contrast to other conditional and unconditional diffusion models, we utilize the idea that the distributions of clean and corrupted images are much closer to each other than each is to the usual Gaussian prior of the reverse process in diffusion models. Our approach therefore requires only low levels of added noise and needs comparatively few sampling steps even without further optimizations. We show that DriftRec naturally generalizes to realistic and difficult scenarios such as unaligned double JPEG compression and blind restoration of JPEGs found online, without having encountered such examples during training.