From Sycophancy to Sensemaking: Premise Governance for Human-AI Decision Making

TL;DR

This paper proposes a premise governance framework for human-AI decision making that emphasizes collaborative negotiation over knowledge, aiming to improve reliability and trustworthiness in complex, uncertain scenarios.

Contribution

It introduces a discrepancy-driven control loop and commitment gating mechanisms to shift from answer generation to collaborative premise management in human-AI partnerships.

Findings

Framework enhances trust through auditable premises.

Discrepancy detection improves alignment in decision-critical contexts.

Proposed evaluation criteria enable falsifiable assessment.

Abstract

As LLMs expand from assistance to decision support, a dangerous pattern emerges: fluent agreement without calibrated judgment. Low-friction assistants can become sycophantic, baking in implicit assumptions and pushing verification costs onto experts, while outcomes arrive too late to serve as reward signals. In deep-uncertainty decisions (where objectives are contested and reversals are costly), scaling fluent agreement amplifies poor commitments faster than it builds expertise. We argue reliable human-AI partnership requires a shift from answer generation to collaborative premise governance over a knowledge substrate, negotiating only what is decision-critical. A discrepancy-driven control loop operates over this substrate: detecting conflicts, localizing misalignment via typed discrepancies (teleological, epistemic, procedural), and triggering bounded negotiation through decision slices. Commitment gating blocks action on uncommitted load-bearing premises unless overridden under logged risk; value-gated challenge allocates probing under interaction cost. Trust then attaches to auditable premises and evidence standards, not conversational fluency. We illustrate with tutoring and propose falsifiable evaluation criteria.