Beyond Words: Evaluating and Bridging Epistemic Divergence in User-Agent Interaction via Theory of Mind

TL;DR

This paper introduces a new benchmark and dataset to evaluate and improve Large Language Models' ability to understand and resolve epistemic divergence in user interactions using Theory of Mind, enhancing their practical interaction capabilities.

Contribution

It formalizes ToM for LLMs as a tool for epistemic divergence detection and proposes a benchmark and dataset to improve models' understanding of user beliefs in real-world tasks.

Findings

Models struggle to identify cognitive gaps affecting task success.

Training on belief tracking data improves reasoning about user mental states.

Enhanced ToM understanding leads to better downstream task performance.

Abstract

Large Language Models (LLMs) have developed rapidly and are widely applied to both general-purpose and professional tasks to assist human users. However, they still struggle to comprehend and respond to the true user needs when intentions and instructions are imprecisely conveyed, leading to a divergence between subjective user believes and true environment states. Resolving this epistemic divergence requires Theory of Mind (ToM), yet existing ToM evaluations for LLMs primarily focus on isolated belief inference, overlooking its functional utility in real-world interaction. To this end, we formalize ToM for LLMs as a mechanism for epistemic divergence detection and resolution, and propose a benchmark, \benchname, to assess how models reconcile user beliefs and profiles in practice. Results across 11 leading models reveal a significant limitation to identify underlying cognitive gaps that impede task success. To bridge this gap, we further curate a trajectory-based ToM dataset linking belief tracking with task-related state inference. The model trained on this data via reinforcement learning shows consistent improvement in reasoning about user mental states, leading to enhanced downstream performance. Our work highlights the practical value of ToM as an essential interaction-level mechanism rather than as a standalone reasoning skill.