LLMs Encode How Difficult Problems Are

TL;DR

This paper investigates whether large language models internally encode problem difficulty similarly to human judgment and how this affects their learning and performance, revealing that human-labeled difficulty correlates strongly with model behavior while automated estimates do not.

Contribution

The study demonstrates that human-annotated difficulty is reliably encoded in LLMs and can guide training to improve accuracy, unlike automated difficulty measures which become misaligned as models improve.

Findings

Human difficulty labels are strongly linearly decodable from LLMs.

Model performance improves when steering towards 'easier' problem representations.

Automated difficulty estimates degrade and misalign during model training.

Abstract

Large language models exhibit a puzzling inconsistency: they solve complex problems yet frequently fail on seemingly simpler ones. We investigate whether LLMs internally encode problem difficulty in a way that aligns with human judgment, and whether this representation tracks generalization during reinforcement learning post-training. We train linear probes across layers and token positions on 60 models, evaluating on mathematical and coding subsets of Easy2HardBench. We find that human-labeled difficulty is strongly linearly decodable (AMC: $\rho \approx 0.88$) and exhibits clear model-size scaling, whereas LLM-derived difficulty is substantially weaker and scales poorly. Steering along the difficulty direction reveals that pushing models toward "easier" representations reduces hallucination and improves accuracy. During GRPO training on Qwen2.5-Math-1.5B, the human-difficulty probe strengthens and positively correlates with test accuracy across training steps, while the LLM-difficulty probe degrades and negatively correlates with performance. These results suggest that human annotations provide a stable difficulty signal that RL amplifies, while automated difficulty estimates derived from model performance become misaligned precisely as models improve. We release probe code and evaluation scripts to facilitate replication.