Crisis-Bench: Benchmarking Strategic Ambiguity and Reputation Management in Large Language Models

TL;DR

Crisis-Bench is a new benchmark that evaluates large language models' ability to manage reputation and strategic ambiguity in high-stakes corporate crises, highlighting the gap between general safety and professional utility.

Contribution

It introduces Crisis-Bench, a multi-agent POMDP framework with a novel economic incentive metric, to assess LLMs' reputation management in complex crisis scenarios.

Findings

Some models exhibit Machiavellian strategic withholding.

Models vary in their ability to balance ethics and strategic ambiguity.

The benchmark reveals a gap between general safety and professional utility.

Abstract

Standard safety alignment optimizes Large Language Models (LLMs) for universal helpfulness and honesty, effectively instilling a rigid "Boy Scout" morality. While robust for general-purpose assistants, this one-size-fits-all ethical framework imposes a "transparency tax" on professional domains requiring strategic ambiguity and information withholding, such as public relations, negotiation, and crisis management. To measure this gap between general safety and professional utility, we introduce Crisis-Bench, a multi-agent Partially Observable Markov Decision Process (POMDP) that evaluates LLMs in high-stakes corporate crises. Spanning 80 diverse storylines across 8 industries, Crisis-Bench tasks an LLM-based Public Relations (PR) Agent with navigating a dynamic 7-day corporate crisis simulation while managing strictly separated Private and Public narrative states to enforce rigorous information asymmetry. Unlike traditional benchmarks that rely on static ground truths, we introduce the Adjudicator-Market Loop: a novel evaluation metric where public sentiment is adjudicated and translated into a simulated stock price, creating a realistic economic incentive structure. Our results expose a critical dichotomy: while some models capitulate to ethical concerns, others demonstrate the capacity for Machiavellian, legitimate strategic withholding in order to stabilize the simulated stock price. Crisis-Bench provides the first quantitative framework for assessing "Reputation Management" capabilities, arguing for a shift from rigid moral absolutism to context-aware professional alignment.