NTIRE 2021 Challenge on Perceptual Image Quality Assessment

TL;DR

This paper details the NTIRE 2021 challenge focused on perceptual image quality assessment, emphasizing the evaluation of GAN-based image distortions with new datasets and demonstrating significant advancements over existing methods.

Contribution

It introduces a new IQA challenge with datasets containing GAN-processed images and subjective scores, fostering development of more effective perceptual quality assessment methods.

Findings

Most submitted models outperform existing IQA methods.

The winning model achieves state-of-the-art performance.

The challenge advances IQA research for GAN-based distortions.

Abstract

This paper reports on the NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021. As a new type of image processing technology, perceptual image processing algorithms based on Generative Adversarial Networks (GAN) have produced images with more realistic textures. These output images have completely different characteristics from traditional distortions, thus pose a new challenge for IQA methods to evaluate their visual quality. In comparison with previous IQA challenges, the training and testing datasets in this challenge include the outputs of perceptual image processing algorithms and the corresponding subjective scores. Thus they can be used to develop and evaluate IQA methods on GAN-based distortions. The challenge has 270 registered participants in total. In the final testing stage, 13 participating teams submitted their models and fact sheets. Almost all of them have achieved much better results than existing IQA methods, while the winning method can demonstrate state-of-the-art performance.