Prompt-SID: Learning Structural Representation Prompt via Latent Diffusion for Single-Image Denoising

TL;DR

Prompt-SID introduces a self-supervised, prompt-learning framework for single-image denoising that preserves structural details using latent diffusion and a structural attention module, outperforming existing methods.

Contribution

It proposes a novel prompt-learning approach with a structural representation generation model and scale replay training for improved denoising performance.

Findings

Effective on synthetic and real-world datasets

Preserves structural details better than existing methods

Achieves state-of-the-art denoising results

Abstract

Many studies have concentrated on constructing supervised models utilizing paired datasets for image denoising, which proves to be expensive and time-consuming. Current self-supervised and unsupervised approaches typically rely on blind-spot networks or sub-image pairs sampling, resulting in pixel information loss and destruction of detailed structural information, thereby significantly constraining the efficacy of such methods. In this paper, we introduce Prompt-SID, a prompt-learning-based single image denoising framework that emphasizes preserving of structural details. This approach is trained in a self-supervised manner using downsampled image pairs. It captures original-scale image information through structural encoding and integrates this prompt into the denoiser. To achieve this, we propose a structural representation generation model based on the latent diffusion process and design a structural attention module within the transformer-based denoiser architecture to decode the prompt. Additionally, we introduce a scale replay training mechanism, which effectively mitigates the scale gap from images of different resolutions. We conduct comprehensive experiments on synthetic, real-world, and fluorescence imaging datasets, showcasing the remarkable effectiveness of Prompt-SID. Our code will be released at https://github.com/huaqlili/Prompt-SID.