KonIQ-10k: Towards an ecologically valid and large-scale IQA database

TL;DR

The paper introduces KonIQ-10k, a large-scale, ecologically valid image quality assessment database created through scalable crowdsourcing, enabling more effective deep learning models for in-the-wild image quality prediction.

Contribution

It presents a novel systematic approach to build large, diverse IQA datasets with extensive crowdsourcing, addressing the resource challenge in creating authentic image quality datasets.

Findings

KonIQ-10k contains 10,073 images with 1.2 million quality ratings.

The dataset demonstrates high ecological validity and diversity.

Crowdsourcing yields reliable quality ratings for large-scale IQA data.

Abstract

The main challenge in applying state-of-the-art deep learning methods to predict image quality in-the-wild is the relatively small size of existing quality scored datasets. The reason for the lack of larger datasets is the massive resources required in generating diverse and publishable content. We present a new systematic and scalable approach to create large-scale, authentic and diverse image datasets for Image Quality Assessment (IQA). We show how we built an IQA database, KonIQ-10k, consisting of 10,073 images, on which we performed very large scale crowdsourcing experiments in order to obtain reliable quality ratings from 1,467 crowd workers (1.2 million ratings). We argue for its ecological validity by analyzing the diversity of the dataset, by comparing it to state-of-the-art IQA databases, and by checking the reliability of our user studies.