Statistical evaluation of visual quality metrics for image denoising

TL;DR

This paper evaluates existing visual quality metrics for denoised images, introduces a new complex dataset, and proposes a novel metric that better correlates with human perception, especially for low contrast and noise-like textures.

Contribution

It presents a new FLT dataset for assessing denoising quality and introduces a new visual quality metric with higher correlation to human opinion.

Findings

Existing metrics have low correlation with human scores on the FLT dataset.

The proposed metric achieves a higher SROCC (up to 0.82) with human opinion scores.

The FLT dataset is more challenging and representative for denoising quality assessment.

Abstract

This paper studies the problem of full reference visual quality assessment of denoised images with a special emphasis on images with low contrast and noise-like texture. Denoising of such images together with noise removal often results in image details loss or smoothing. A new test image database, FLT, containing 75 noise-free "reference" images and 300 filtered ("distorted") images is developed. Each reference image, corrupted by an additive white Gaussian noise, is denoised by the BM3D filter with four different values of threshold parameter (four levels of noise suppression). After carrying out a perceptual quality assessment of distorted images, the mean opinion scores (MOS) are obtained and compared with the values of known full reference quality metrics. As a result, the Spearman Rank Order Correlation Coefficient (SROCC) between PSNR values and MOS has a value close to zero, and SROCC between values of known full-reference image visual quality metrics and MOS does not exceed 0.82 (which is reached by a new visual quality metric proposed in this paper). The FLT dataset is more complex than earlier datasets used for assessment of visual quality for image denoising. Thus, it can be effectively used to design new image visual quality metrics for image denoising.