KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment

TL;DR

This paper introduces KonIQ-10k, the largest ecologically valid image quality assessment dataset created through crowdsourcing, and proposes a deep learning model, KonCept512, that achieves state-of-the-art performance in blind IQA.

Contribution

It presents the first large-scale, ecologically valid IQA dataset with extensive crowdsourced ratings and a novel deep learning model demonstrating superior generalization.

Findings

KonIQ-10k contains 10,073 images with 1.2 million quality ratings.

KonCept512 achieves a 0.921 SROCC on the dataset, surpassing previous models.

The model performs comparably to having nine subjective scores per image.

Abstract

Deep learning methods for image quality assessment (IQA) are limited due to the small size of existing datasets. Extensive datasets require substantial resources both for generating publishable content and annotating it accurately. We present a systematic and scalable approach to creating KonIQ-10k, the largest IQA dataset to date, consisting of 10,073 quality scored images. It is the first in-the-wild database aiming for ecological validity, concerning the authenticity of distortions, the diversity of content, and quality-related indicators. Through the use of crowdsourcing, we obtained 1.2 million reliable quality ratings from 1,459 crowd workers, paving the way for more general IQA models. We propose a novel, deep learning model (KonCept512), to show an excellent generalization beyond the test set (0.921 SROCC), to the current state-of-the-art database LIVE-in-the-Wild (0.825 SROCC). The model derives its core performance from the InceptionResNet architecture, being trained at a higher resolution than previous models (512x384). Correlation analysis shows that KonCept512 performs similar to having 9 subjective scores for each test image.