Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions

TL;DR

This paper identifies a persistent gap between what large language models claim as their values and their actual behavior, introducing a framework to measure and address this misalignment.

Contribution

The paper introduces VALDI, a comprehensive framework with metrics and scenarios to systematically evaluate value-action alignment in LLMs, and proposes VIVALDI for intervention.

Findings

LLMs often misalign expressed values with generated dialogue

VALDI effectively quantifies value adherence across multiple domains

VIVALDI shows potential for improving value alignment through multi-agent auditing

Abstract

Large language models (LLMs) are often evaluated based on their stated values, yet these do not reliably translate into their actions, a discrepancy termed "value-action gap." In this work, we argue that this gap persists even under explicit reasoning, revealing a deeper failure mode we call "Pseudo-Deliberation": the appearance of principled reasoning without corresponding behavioral alignment. To study this systematically, we introduce VALDI, a framework for measuring alignment between stated values and generated dialogue. VALDI includes 4,941 human-centered scenarios across five domains, three tasks that elicit value articulation, reasoning, and action, and five metrics for quantifying value adherence. Across both proprietary and open-source LLMs, we observe consistent misalignment between expressed values and downstream dialogues. To investigate intervention strategies, we propose VIVALDI, a multi-agent value auditor that intervenes at different stages of generation.