Who Sees What? Structured Thought-Action Sequences for Epistemic Reasoning in LLMs

TL;DR

This paper explores how structured thought-action sequences can enhance epistemic reasoning in large language models, revealing limitations and the need for more explicit belief and cost modeling for better perspective-taking.

Contribution

It introduces a structured solution-processing pipeline using transformed solution graphs to generate thought-action examples for LLMs, highlighting their limited effectiveness in complex perspective-taking tasks.

Findings

L-type examples slightly reduce clarification requests

Agents succeed in basic attentional filtering tasks

Structured examples alone are insufficient for complex perspective-taking

Abstract

Recent advances in large language models (LLMs) and reasoning frameworks have opened new possibilities for improving the perspective -taking capabilities of autonomous agents. However, tasks that involve active perception, collaborative reasoning, and perspective taking (understanding what another agent can see or knows) pose persistent challenges for current LLM-based systems. This study investigates the potential of structured examples derived from transformed solution graphs generated by the Fast Downward planner to improve the performance of LLM-based agents within a ReAct framework. We propose a structured solution-processing pipeline that generates three distinct categories of examples: optimal goal paths (G-type), informative node paths (E-type), and step-by-step optimal decision sequences contrasting alternative actions (L-type). These solutions are further converted into ``thought-action'' examples by prompting an LLM to explicitly articulate the reasoning behind each decision. While L-type examples slightly reduce clarification requests and overall action steps, they do not yield consistent improvements. Agents are successful in tasks requiring basic attentional filtering but struggle in scenarios that required mentalising about occluded spaces or weighing the costs of epistemic actions. These findings suggest that structured examples alone are insufficient for robust perspective-taking, underscoring the need for explicit belief tracking, cost modelling, and richer environments to enable socially grounded collaboration in LLM-based agents.