Let's Put Ourselves in Sally's Shoes: Shoes-of-Others Prefilling Improves Theory of Mind in Large Language Models

TL;DR

This paper introduces a new inference-time method called Shoes-of-Others prefilling that enhances Theory of Mind capabilities in large language models across various contexts by prompting models to adopt perspectives of specific characters.

Contribution

The study proposes a novel, broadly applicable prefilling technique for LLMs that improves ToM performance without requiring fine-tuning or assumptions about world state changes.

Findings

Consistently improves ToM across five mental state categories

Effective in conversational and narrative contexts

Enhances faithfulness of generated thoughts

Abstract

Recent studies have shown that Theory of Mind (ToM) in large language models (LLMs) has not reached human-level performance yet. Since fine-tuning LLMs on ToM datasets often degrades their generalization, several inference-time methods have been proposed to enhance ToM in LLMs. However, existing inference-time methods for ToM are specialized for inferring beliefs from contexts involving changes in the world state. In this study, we present a new inference-time method for ToM, Shoes-of-Others (SoO) prefilling, which makes fewer assumptions about contexts and is applicable to broader scenarios. SoO prefilling simply specifies the beginning of LLM outputs with ``Let's put ourselves in A's shoes.'', where A denotes the target character's name. We evaluate SoO prefilling on two benchmarks that assess ToM in conversational and narrative contexts without changes in the world state and find that it consistently improves ToM across five categories of mental states. Our analysis suggests that SoO prefilling elicits faithful thoughts, thereby improving the ToM performance.