Explanation Quality Assessment as Ranking with Listwise Rewards

TL;DR

This paper proposes a ranking-based approach to evaluate explanation quality, demonstrating that ranking losses outperform regression, especially with high-quality data, and enabling stable policy optimization.

Contribution

It introduces a novel ranking formulation for explanation assessment, showing improved performance over traditional regression methods and highlighting the importance of data quality.

Findings

Ranking losses outperform regression in explanation scoring.

Listwise objectives excel with well-separated quality levels.

Small models can match larger ones when trained on high-quality data.

Abstract

We reformulate explanation quality assessment as a ranking problem rather than a generation problem. Instead of optimizing models to produce a single "best" explanation token-by-token, we train reward models to discriminate among multiple candidate explanations and learn their relative quality. Concretely, we construct per-instance candidate sets with graded quality levels and train listwise and pairwise ranking models (ListNet, LambdaRank, RankNet) to preserve ordinal structure and avoid score compression typical of pointwise regression or binary preference objectives. We observe three findings: First, ranking losses consistently outperform regression on score separation across all domains tested. Second, the optimal ranking loss depends on data characteristics: listwise objectives excel with well-separated quality tiers, while pairwise methods are more robust to noisy natural annotations. Third, when trained on carefully curated and well-structured data, small encoder models can match models that are orders of magnitude larger, suggesting that data quality matters more than model scale. Finally, when used as rewards in policy optimization, ranking-based scores enable stable convergence in settings where regression-based rewards fail entirely. Code and data are available at: https://github.com/Tankiit/PPO_Learning_to_rank