Beyond Unidimensionality: General Factors and Residual Heterogeneity in Performance Evaluation

TL;DR

This study investigates how expert performance evaluations in soccer condense complex, multidimensional data into summary ratings, revealing a dominant general factor alongside meaningful residual dimensions, informing better measurement system design.

Contribution

It provides a detailed analysis of the dimensional structure of expert evaluations, demonstrating the coexistence of a strong general factor and stable residual components, with implications for evaluation system design.

Findings

The first principal component explains 40.6% of attribute variance.

A comprehensive model with all attributes predicts overall ratings with R^2 = 0.814.

Evaluation structures include a dominant general factor plus meaningful residual dimensions.

Abstract

How do evaluation systems compress multidimensional performance information into summary ratings? Using expert assessments of 9,669 professional soccer players on 28 attributes, we characterize the dimensional structure of evaluation outputs. The first principal component explains 40.6% of attribute variance, indicating a strong general factor, but formal noise discrimination procedures retain four components and bootstrap resampling confirms that this structure is highly stable. Internal consistency is high without evidence of redundancy. In out of sample prediction of expert overall ratings, a comprehensive model using the full attribute set substantially outperforms a single-factor summary (cross-validated R squared = 0.814). Overall, performance evaluations exhibit moderate information compression; they combine shared variance with stable residual dimensions that are economically meaningful for evaluation outcomes, with direct implications for the design of measurement systems.