Position: Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered

TL;DR

This paper advocates for shifting the evaluation of visual processing systems from traditional metric-based benchmarks to more human-centered, perception-aware approaches to better align with user preferences and foster innovation.

Contribution

It highlights the limitations of current image quality metrics and proposes a rebalanced evaluation paradigm emphasizing human perception and context-awareness.

Findings

Objective IQA metrics diverge from human perception

Current metrics may constrain innovation in visual processing

A call for more human-centered evaluation methods

Abstract

This position paper argues that the evaluation of modern visual processing systems should no longer be driven primarily by single-metric image quality assessment benchmarks, particularly in the era of generative and perception-oriented methods. Image restoration exemplifies this divergence: while objective IQA metrics enable reproducible, scalable evaluation, they have increasingly drifted apart from human perception and user preferences. We contend that this mismatch risks constraining innovation and misguiding research progress across visual processing tasks. Rather than rejecting metrics altogether, this paper calls for a rebalancing of evaluation paradigms, advocating a more human-centered, context-aware, and fine-grained approach to assessing the visual models' outcomes.