MMP-2K: A Benchmark Multi-Labeled Macro Photography Image Quality Assessment Database

TL;DR

This paper introduces MMP-2K, a large-scale, multi-labeled benchmark database for macro photography image quality assessment, addressing the lack of specialized datasets and revealing the limitations of existing IQA metrics on macro images.

Contribution

The paper presents the first large-scale, multi-labeled MPIQA database, MMP-2K, with diverse content and detailed quality reports, facilitating improved macro image quality assessment.

Findings

State-of-the-art IQA metrics underperform on macro images.

The MMP-2K database enables better evaluation and development of MPIQA methods.

Diverse macro images and detailed quality reports support comprehensive analysis.

Abstract

Macro photography (MP) is a specialized field of photography that captures objects at an extremely close range, revealing tiny details. Although an accurate macro photography image quality assessment (MPIQA) metric can benefit macro photograph capturing, which is vital in some domains such as scientific research and medical applications, the lack of MPIQA data limits the development of MPIQA metrics. To address this limitation, we conducted a large-scale MPIQA study. Specifically, to ensure diversity both in content and quality, we sampled 2,000 MP images from 15,700 MP images, collected from three public image websites. For each MP image, 17 (out of 21 after outlier removal) quality ratings and a detailed quality report of distortion magnitudes, types, and positions are gathered by a lab study. The images, quality ratings, and quality reports form our novel multi-labeled MPIQA database, MMP-2k. Experimental results showed that the state-of-the-art generic IQA metrics underperform on MP images. The database and supplementary materials are available at https://github.com/Future-IQA/MMP-2k.