Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisition

TL;DR

This paper investigates the impact of Theory of Mind modelling on dialogue-based collaborative plan acquisition, revealing that its benefits diminish when predicting missing knowledge, and questioning its actual role in effective collaboration.

Contribution

The study provides a systematic comparison of models with and without ToM features, showing that learned ToM features may not reflect true mental state understanding in CPA tasks.

Findings

Performance nearly doubles when predicting one's own missing knowledge.

ToM features do not significantly improve model performance over baseline methods.

Learned ToM features may capture data patterns unrelated to actual mental state modeling.

Abstract

Recent work on dialogue-based collaborative plan acquisition (CPA) has suggested that Theory of Mind (ToM) modelling can improve missing knowledge prediction in settings with asymmetric skill-sets and knowledge. Although ToM was claimed to be important for effective collaboration, its real impact on this novel task remains under-explored. By representing plans as graphs and by exploiting task-specific constraints we show that, as performance on CPA nearly doubles when predicting one's own missing knowledge, the improvements due to ToM modelling diminish. This phenomenon persists even when evaluating existing baseline methods. To better understand the relevance of ToM for CPA, we report a principled performance comparison of models with and without ToM features. Results across different models and ablations consistently suggest that learned ToM features are indeed more likely to reflect latent patterns in the data with no perceivable link to ToM. This finding calls for a deeper understanding of the role of ToM in CPA and beyond, as well as new methods for modelling and evaluating mental states in computational collaborative agents.