Towards Real-World Video Denosing: A Practical Video Denosing Dataset and Network

TL;DR

This paper introduces PVDD, a new realistic video denoising dataset with dynamic scenes and real noise, and proposes RVDT, a transformer-based model that achieves state-of-the-art results on PVDD and other benchmarks.

Contribution

The paper presents a comprehensive real-world video denoising dataset and a novel transformer-based denoising network that outperforms existing methods.

Findings

Models trained on PVDD outperform those trained on other datasets.

RVDT achieves superior denoising performance compared to other networks.

PVDD enables better generalization to real-world videos.

Abstract

To facilitate video denoising research, we construct a compelling dataset, namely, "Practical Video Denoising Dataset" (PVDD), containing 200 noisy-clean dynamic video pairs in both sRGB and RAW format. Compared with existing datasets consisting of limited motion information, PVDD covers dynamic scenes with varying and natural motion. Different from datasets using primarily Gaussian or Poisson distributions to synthesize noise in the sRGB domain, PVDD synthesizes realistic noise from the RAW domain with a physically meaningful sensor noise model followed by ISP processing. Moreover, we also propose a new video denoising framework, called Recurrent Video Denoising Transformer (RVDT), which can achieve SOTA performance on PVDD and other current video denoising benchmarks. RVDT consists of both spatial and temporal transformer blocks to conduct denoising with long-range operations on the spatial dimension and long-term propagation on the temporal dimension. Especially, RVDT exploits the attention mechanism to implement the bi-directional feature propagation with both implicit and explicit temporal modeling. Extensive experiments demonstrate that 1) models trained on PVDD achieve superior denoising performance on many challenging real-world videos than on models trained on other existing datasets; 2) trained on the same dataset, our proposed RVDT can have better denoising performance than other types of networks.