Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training

TL;DR

This paper introduces PNGAN, a novel noise-aware GAN framework that generates realistic noisy images to improve denoising models, reducing the need for large real noisy-clean datasets.

Contribution

The work proposes a pixel-level adversarial training framework and an efficient generator architecture for realistic noise synthesis, enhancing denoising performance.

Findings

Generated noise closely matches real noise in intensity and distribution

Denoising models trained on generated noise achieve SOTA results

The framework reduces dependency on large real noisy-clean datasets

Abstract

Existing deep learning real denoising methods require a large amount of noisy-clean image pairs for supervision. Nonetheless, capturing a real noisy-clean dataset is an unacceptable expensive and cumbersome procedure. To alleviate this problem, this work investigates how to generate realistic noisy images. Firstly, we formulate a simple yet reasonable noise model that treats each real noisy pixel as a random variable. This model splits the noisy image generation problem into two sub-problems: image domain alignment and noise domain alignment. Subsequently, we propose a novel framework, namely Pixel-level Noise-aware Generative Adversarial Network (PNGAN). PNGAN employs a pre-trained real denoiser to map the fake and real noisy images into a nearly noise-free solution space to perform image domain alignment. Simultaneously, PNGAN establishes a pixel-level adversarial training to conduct noise domain alignment. Additionally, for better noise fitting, we present an efficient architecture Simple Multi-scale Network (SMNet) as the generator. Qualitative validation shows that noise generated by PNGAN is highly similar to real noise in terms of intensity and distribution. Quantitative experiments demonstrate that a series of denoisers trained with the generated noisy images achieve state-of-the-art (SOTA) results on four real denoising benchmarks. Part of codes, pre-trained models, and results are available at https://github.com/caiyuanhao1998/PNGAN for comparisons.