Q-Bench-Portrait: Benchmarking Multimodal Large Language Models on Portrait Image Quality Perception

TL;DR

This paper introduces Q-Bench-Portrait, a comprehensive benchmark for evaluating multimodal large language models' ability to perceive and assess portrait image quality across various distortions and aesthetic dimensions.

Contribution

It presents the first dedicated benchmark for portrait image quality perception, including diverse image types, quality dimensions, and question formats, enabling systematic evaluation of MLLMs.

Findings

Current MLLMs show limited and imprecise portrait perception.

Models perform significantly below human judgment levels.

Benchmark reveals substantial gaps in model capabilities.

Abstract

Recent advances in multimodal large language models (MLLMs) have demonstrated impressive performance on existing low-level vision benchmarks, which primarily focus on generic images. However, their capabilities to perceive and assess portrait images, a domain characterized by distinct structural and perceptual properties, remain largely underexplored. To this end, we introduce Q-Bench-Portrait, the first holistic benchmark specifically designed for portrait image quality perception, comprising 2,765 image-question-answer triplets and featuring (1) diverse portrait image sources, including natural, synthetic distortion, AI-generated, artistic, and computer graphics images; (2) comprehensive quality dimensions, covering technical distortions, AIGC-specific distortions, and aesthetics; and (3) a range of question formats, including single-choice, multiple-choice, true/false, and open-ended questions, at both global and local levels. Based on Q-Bench-Portrait, we evaluate 20 open-source and 5 closed-source MLLMs, revealing that although current models demonstrate some competence in portrait image perception, their performance remains limited and imprecise, with a clear gap relative to human judgments. We hope that the proposed benchmark will foster further research into enhancing the portrait image perception capabilities of both general-purpose and domain-specific MLLMs.