Moral Mazes in the Era of LLMs

TL;DR

This study examines how large language models (LLMs) can navigate complex workplace social norms through a simulated email task, revealing systematic differences from humans and potential for reshaping professional communication.

Contribution

The paper introduces HR Simulator, a novel benchmark for evaluating LLMs in workplace social scenarios, and analyzes their communication style, performance, and emergent norms compared to humans.

Findings

LLM emails are more formal and empathetic than human emails.

Humans underperform compared to LLMs in scenario pass rates.

Rewritten human emails by LLMs can outperform both humans and original LLMs.

Abstract

Navigating complex social situations is an integral part of corporate life, ranging from giving critical feedback without hurting morale to rejecting requests without alienating teammates. Although large language models (LLMs) are permeating the workplace, it is unclear how well they can navigate these norms. To investigate this question, we created HR Simulator, a game where users roleplay as an HR officer and write emails to tackle challenging workplace scenarios, evaluated with GPT-4o as a judge based on scenario-specific rubrics. We analyze over 600 human and LLM emails and find systematic differences in style: LLM emails are more formal and empathetic. Furthermore, humans underperform LLMs (e.g., 23.5% vs. 48-54% scenario pass rate), but human emails rewritten by LLMs can outperform both, which indicates a hybrid advantage. On the evaluation side, judges can exhibit differences in their email preferences: an analysis of 10 judge models reveals evidence for emergent tact, where weaker models prefer direct, blunt communication but stronger models prefer more subtle messages. Judges also agree with each other more as they scale, which hints at a convergence toward shared communicative norms that may differ from humans'. Overall, our results suggest LLMs could substantially reshape communication in the workplace if they are widely adopted in professional correspondence.