Evaluating Unsupervised Denoising Requires Unsupervised Metrics

TL;DR

This paper introduces two unsupervised metrics for evaluating denoising quality directly from noisy data, enabling assessment without ground-truth images, which is crucial for real-world applications.

Contribution

The authors propose and theoretically validate two novel unsupervised metrics, MSE and PSNR, for evaluating denoising performance without clean reference images.

Findings

Metrics accurately estimate supervised MSE and PSNR asymptotically.

Validated on synthetic noise data, showing practical accuracy.

Effective in real-world imaging modalities like microscopy and raw videos.

Abstract

Unsupervised denoising is a crucial challenge in real-world imaging applications. Unsupervised deep-learning methods have demonstrated impressive performance on benchmarks based on synthetic noise. However, no metrics are available to evaluate these methods in an unsupervised fashion. This is highly problematic for the many practical applications where ground-truth clean images are not available. In this work, we propose two novel metrics: the unsupervised mean squared error (MSE) and the unsupervised peak signal-to-noise ratio (PSNR), which are computed using only noisy data. We provide a theoretical analysis of these metrics, showing that they are asymptotically consistent estimators of the supervised MSE and PSNR. Controlled numerical experiments with synthetic noise confirm that they provide accurate approximations in practice. We validate our approach on real-world data from two imaging modalities: videos in raw format and transmission electron microscopy. Our results demonstrate that the proposed metrics enable unsupervised evaluation of denoising methods based exclusively on noisy data.