Exploiting Diffusion Prior for Task-driven Image Restoration

TL;DR

This paper introduces EDTR, a novel method that leverages diffusion priors and pre-restored images to improve task-driven image restoration, especially under complex degradations, enhancing both task performance and visual quality.

Contribution

The paper presents a new approach that directly incorporates diffusion priors into TDIR, effectively restoring task-relevant details from heavily degraded images.

Findings

Significantly improves task performance across diverse image restoration tasks.

Enhances visual quality of restored images with complex degradations.

Utilizes fewer denoising steps to focus on task-relevant details.

Abstract

Task-driven image restoration (TDIR) has recently emerged to address performance drops in high-level vision tasks caused by low-quality (LQ) inputs. Previous TDIR methods struggle to handle practical scenarios in which images are degraded by multiple complex factors, leaving minimal clues for restoration. This motivates us to leverage the diffusion prior, one of the most powerful natural image priors. However, while the diffusion prior can help generate visually plausible results, using it to restore task-relevant details remains challenging, even when combined with recent TDIR methods. To address this, we propose EDTR, which effectively harnesses the power of diffusion prior to restore task-relevant details. Specifically, we propose directly leveraging useful clues from LQ images in the diffusion process by generating from pixel-error-based pre-restored LQ images with mild noise added. Moreover, we employ a small number of denoising steps to prevent the generation of redundant details that dilute crucial task-related information. We demonstrate that our method effectively utilizes diffusion prior for TDIR, significantly enhancing task performance and visual quality across diverse tasks with multiple complex degradations.