Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Task

TL;DR

This paper introduces MindGames, a new task to evaluate whether Large Language Models can perform planning-based Theory of Mind, revealing a significant gap compared to human social reasoning abilities.

Contribution

The paper presents MindGames, a novel multi-step persuasion task explicitly designed to assess planning and mental state inference in LLMs, highlighting their limitations.

Findings

Humans outperform o1-preview by 11% on the PToM task.

o1-preview excels over humans in a planning task with minimal mental inference.

Results reveal a gap between LLMs and humans in social reasoning and mental state understanding.

Abstract

Recent evidence suggests Large Language Models (LLMs) display Theory of Mind (ToM) abilities. Most ToM experiments place participants in a spectatorial role, wherein they predict and interpret other agents' behavior. However, human ToM also contributes to dynamically planning action and strategically intervening on others' mental states. We present MindGames: a novel `planning theory of mind' (PToM) task which requires agents to infer an interlocutor's beliefs and desires to persuade them to alter their behavior. Unlike previous evaluations, we explicitly evaluate use cases of ToM. We find that humans significantly outperform o1-preview (an LLM) at our PToM task (11% higher; $p=0.006$). We hypothesize this is because humans have an implicit causal model of other agents (e.g., they know, as our task requires, to ask about people's preferences). In contrast, o1-preview outperforms humans in a baseline condition which requires a similar amount of planning but minimal mental state inferences (e.g., o1-preview is better than humans at planning when already given someone's preferences). These results suggest a significant gap between human-like social reasoning and LLM abilities.