UHD-IQA Benchmark Database: Pushing the Boundaries of Blind Photo Quality Assessment

TL;DR

This paper presents a new UHD-1 image quality assessment dataset with high-quality, diverse images and reliable crowdsourced annotations, aimed at advancing blind photo quality evaluation methods.

Contribution

The creation of a large, curated UHD-1 IQA dataset focused on high-quality aesthetic images with expert-verified annotations and rich metadata, filling a gap in existing datasets.

Findings

Dataset includes 6073 images with 20 reliable ratings each.

Rich metadata and annotations enable diverse research applications.

Focus on high-quality, non-synthetic images enhances model relevance.

Abstract

We introduce a novel Image Quality Assessment (IQA) dataset comprising 6073 UHD-1 (4K) images, annotated at a fixed width of 3840 pixels. Contrary to existing No-Reference (NR) IQA datasets, ours focuses on highly aesthetic photos of high technical quality, filling a gap in the literature. The images, carefully curated to exclude synthetic content, are sufficiently diverse to train general NR-IQA models. Importantly, the dataset is annotated with perceptual quality ratings obtained through a crowdsourcing study. Ten expert raters, comprising photographers and graphics artists, assessed each image at least twice in multiple sessions spanning several days, resulting in 20 highly reliable ratings per image. Annotators were rigorously selected based on several metrics, including self-consistency, to ensure their reliability. The dataset includes rich metadata with user and machine-generated tags from over 5,000 categories and popularity indicators such as favorites, likes, downloads, and views. With its unique characteristics, such as its focus on high-quality images, reliable crowdsourced annotations, and high annotation resolution, our dataset opens up new opportunities for advancing perceptual image quality assessment research and developing practical NR-IQA models that apply to modern photos. Our dataset is available at https://database.mmsp-kn.de/uhd-iqa-benchmark-database.html