Distilling Bayesian Belief States into Language Models for Auditable Negotiation

TL;DR

This paper introduces BOND, a framework that distills Bayesian opponent belief states into language models for auditable negotiation, enabling transparent belief inference and decision-making in dialogue agents.

Contribution

BOND combines a Bayesian teacher with a smaller student model to produce interpretable belief states and improve auditable negotiation capabilities over existing methods.

Findings

BOND outperforms state-of-the-art on the CaSiNo dataset with a mean Brier score of 0.085.

The distilled student maintains belief signals with a Brier score of 0.114, better than uniform priors.

Smaller models yield better calibration of elicited posterior beliefs compared to larger baselines.

Abstract

Negotiation agents must infer what their counterpart values, update those beliefs over dialogue turns, and choose actions under uncertainty. End-to-end large language models (LLMs) can imitate negotiation dialogue, but their opponent beliefs are usually implicit and difficult to inspect. We propose BOND (Bayesian Opponent-belief Negotiation Distillation), a framework for auditable negotiation. BOND consists of an LLM-based Bayesian teacher that scores dialogue contexts against the six possible opponent priority orderings, updates a posterior over those orderings, and uses the posterior for menu-based decision making, as well as a smaller 8B student language model that emits both negotiation actions and normalized posterior beliefs as tagged text. In the CaSiNo negotiation dataset, BOND outperforms the state-of-the-art and achieves mean Brier score 0.085 over opponent-priority posteriors. The distilled student preserves much of this belief signal, achieving Brier 0.114, below the uniform six-ordering reference of 5/36, approximately 0.139. Compared with a 70B structured-CoT baseline, the significantly smaller 8B student model yields substantially better elicited posterior calibration. We further showcase auditability through posterior trajectories, belief-versus-policy error decomposition, and posterior-prefix interventions. These diagnostics reveal that distillation preserves a scoreable belief report more strongly than causal belief-conditioned control, making weak belief-action coupling visible, not hidden.