Why Expert Alignment Is Hard: Evidence from Subjective Evaluation

TL;DR

Aligning large language models with expert judgment in subjective tasks is challenging due to expert heterogeneity, tacit criteria, and judgment instability, as shown by empirical evaluation patterns.

Contribution

This paper provides empirical evidence on the complexities of expert alignment, highlighting the heterogeneity and tacit nature of subjective evaluation.

Findings

Alignment difficulty varies across experts.

Explicit criteria do not always improve alignment.

Alignment is easier on content-based dimensions.

Abstract

Aligning large language models with expert judgment is especially difficult in subjective evaluation tasks, where experts may disagree, rely on tacit criteria, and change their judgments over time. In this paper, we study expert alignment as a way to understand this difficulty. Using expert evaluations and follow-up questionnaires, we examine how different forms of expert information affect alignment and what this reveals about subjective judgment. Our findings show four consistent patterns. First, alignment difficulty varies substantially across experts, suggesting that expert evaluation styles differ widely in their distance from a model's prior behavior. Second, explicit criteria and reasoning do not always improve alignment, indicating that expert judgment is not fully captured by verbalized rules. Third, editing is sensitive to both the number and the identity of examples, with small numbers of edits providing useful but unstable gains. Fourth, alignment difficulty differs across evaluation dimensions: dimensions grounded more directly in proposal content are easier to align, while dimensions requiring external knowledge or value-based judgment remain harder. Taken together, these results suggest that expert alignment is difficult not only because of model limitations, but also because subjective evaluation is inherently heterogeneous, partly tacit, dimension-dependent, and temporally unstable.