Deep Semantic Statistics Matching (D2SM) Denoising Network

TL;DR

This paper introduces D2SM, a denoising network that leverages semantic features from pretrained classifiers to better preserve image semantics, improving denoising and high-level vision task performance.

Contribution

It proposes a novel semantic distribution matching approach for image denoising, enhancing semantic consistency and generalization to tasks like super-resolution and dehazing.

Findings

Significant improvement in denoising quality on Cityscapes dataset.

Enhanced semantic segmentation accuracy after denoising.

Potential as a versatile plug-and-play component for various image restoration tasks.

Abstract

The ultimate aim of image restoration like denoising is to find an exact correlation between the noisy and clear image domains. But the optimization of end-to-end denoising learning like pixel-wise losses is performed in a sample-to-sample manner, which ignores the intrinsic correlation of images, especially semantics. In this paper, we introduce the Deep Semantic Statistics Matching (D2SM) Denoising Network. It exploits semantic features of pretrained classification networks, then it implicitly matches the probabilistic distribution of clear images at the semantic feature space. By learning to preserve the semantic distribution of denoised images, we empirically find our method significantly improves the denoising capabilities of networks, and the denoised results can be better understood by high-level vision tasks. Comprehensive experiments conducted on the noisy Cityscapes dataset demonstrate the superiority of our method on both the denoising performance and semantic segmentation accuracy. Moreover, the performance improvement observed on our extended tasks including super-resolution and dehazing experiments shows its potentiality as a new general plug-and-play component.