Subjective Quality Assessment for Images Generated by Computer Graphics

TL;DR

This paper introduces a large-scale database for subjective quality assessment of computer graphics generated images, revealing the inadequacy of existing no-reference IQA methods for CG-IQA tasks and highlighting the need for more effective models.

Contribution

It creates a comprehensive CG-IQA database and evaluates existing no-reference IQA methods, demonstrating their limitations for CG images.

Findings

Handcrafted NR-IQA methods show low correlation with subjective scores.

Deep learning based methods perform relatively better than handcrafted methods.

Current NR-IQA models are insufficient for CG-IQA tasks.

Abstract

With the development of rendering techniques, computer graphics generated images (CGIs) have been widely used in practical application scenarios such as architecture design, video games, simulators, movies, etc. Different from natural scene images (NSIs), the distortions of CGIs are usually caused by poor rending settings and limited computation resources. What's more, some CGIs may also suffer from compression distortions in transmission systems like cloud gaming and stream media. However, limited work has been put forward to tackle the problem of computer graphics generated images' quality assessment (CG-IQA). Therefore, in this paper, we establish a large-scale subjective CG-IQA database to deal with the challenge of CG-IQA tasks. We collect 25,454 in-the-wild CGIs through previous databases and personal collection. After data cleaning, we carefully select 1,200 CGIs to conduct the subjective experiment. Several popular no-reference image quality assessment (NR-IQA) methods are tested on our database. The experimental results show that the handcrafted-based methods achieve low correlation with subjective judgment and deep learning based methods obtain relatively better performance, which demonstrates that the current NR-IQA models are not suitable for CG-IQA tasks and more effective models are urgently needed.