Beyond Image Prior: Embedding Noise Prior into Conditional Denoising Transformer

TL;DR

This paper introduces a novel denoising framework that explicitly estimates noise priors from single images and incorporates them into a transformer model, improving generalization across diverse noise conditions.

Contribution

It proposes a Locally Noise Prior Estimation (LoNPE) algorithm and a Conditional Denoising Transformer that leverage noise priors for enhanced denoising performance.

Findings

Outperforms state-of-the-art methods on synthetic datasets

Effectively handles real-world noisy images

Demonstrates improved generalization across noise types

Abstract

Existing learning-based denoising methods typically train models to generalize the image prior from large-scale datasets, suffering from the variability in noise distributions encountered in real-world scenarios. In this work, we propose a new perspective on the denoising challenge by highlighting the distinct separation between noise and image priors. This insight forms the basis for our development of conditional optimization framework, designed to overcome the constraints of traditional denoising framework. To this end, we introduce a Locally Noise Prior Estimation (LoNPE) algorithm, which accurately estimates the noise prior directly from a single raw noisy image. This estimation acts as an explicit prior representation of the camera sensor's imaging environment, distinct from the image prior of scenes. Additionally, we design an auxiliary learnable LoNPE network tailored for practical application to sRGB noisy images. Leveraging the estimated noise prior, we present a novel Conditional Denoising Transformer (Condformer), by incorporating the noise prior into a conditional self-attention mechanism. This integration allows the Condformer to segment the optimization process into multiple explicit subspaces, significantly enhancing the model's generalization and flexibility. Extensive experimental evaluations on both synthetic and real-world datasets, demonstrate that the proposed method achieves superior performance over current state-of-the-art methods. The source code is available at https://github.com/YuanfeiHuang/Condformer.