Pressure Reveals Character: Behavioural Alignment Evaluation at Depth

TL;DR

This paper introduces a comprehensive multi-turn alignment benchmark for language models, revealing that even top models exhibit significant weaknesses across various behavioral categories under realistic pressure scenarios.

Contribution

The paper presents a new benchmark with 904 scenarios validated by humans, and demonstrates that alignment is a unified trait across different behavioral categories in language models.

Findings

Models show consistent weaknesses across categories.

Alignment behaves as a unified construct similar to a g-factor.

Top models still exhibit notable gaps in specific alignment areas.

Abstract

Evaluating alignment in language models requires testing how they behave under realistic pressure, not just what they claim they would do. While alignment failures increasingly cause real-world harm, comprehensive evaluation frameworks with realistic multi-turn scenarios remain lacking. We introduce an alignment benchmark spanning 904 scenarios across six categories -- Honesty, Safety, Non-Manipulation, Robustness, Corrigibility, and Scheming -- validated as realistic by human raters. Our scenarios place models under conflicting instructions, simulated tool access, and multi-turn escalation to reveal behavioural tendencies that single-turn evaluations miss. Evaluating 24 frontier models using LLM judges validated against human annotations, we find that even top-performing models exhibit gaps in specific categories, while the majority of models show consistent weaknesses across the board. Factor analysis reveals that alignment behaves as a unified construct (analogous to the g-factor in cognitive research) with models scoring high on one category tending to score high on others. We publicly release the benchmark and an interactive leaderboard to support ongoing evaluation, with plans to expand scenarios in areas where we observe persistent weaknesses and to add new models as they are released.