ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind

TL;DR

ToMATO is a novel benchmark for evaluating the Theory of Mind in language models, emphasizing diverse mental states, false beliefs, and personality traits through role-playing conversations and verbalized thoughts.

Contribution

It introduces a new dataset and evaluation method that captures a wide range of mental states and personality influences in LLMs, addressing limitations of existing ToM benchmarks.

Findings

LLMs struggle with false beliefs and personality diversity.

GPT-4o mini performs below human levels in ToM tasks.

The dataset reveals frequent false beliefs due to information asymmetry.

Abstract

Existing Theory of Mind (ToM) benchmarks diverge from real-world scenarios in three aspects: 1) they assess a limited range of mental states such as beliefs, 2) false beliefs are not comprehensively explored, and 3) the diverse personality traits of characters are overlooked. To address these challenges, we introduce ToMATO, a new ToM benchmark formulated as multiple-choice QA over conversations. ToMATO is generated via LLM-LLM conversations featuring information asymmetry. By employing a prompting method that requires role-playing LLMs to verbalize their thoughts before each utterance, we capture both first- and second-order mental states across five categories: belief, intention, desire, emotion, and knowledge. These verbalized thoughts serve as answers to questions designed to assess the mental states of characters within conversations. Furthermore, the information asymmetry introduced by hiding thoughts from others induces the generation of false beliefs about various mental states. Assigning distinct personality traits to LLMs further diversifies both utterances and thoughts. ToMATO consists of 5.4k questions, 753 conversations, and 15 personality trait patterns. Our analysis shows that this dataset construction approach frequently generates false beliefs due to the information asymmetry between role-playing LLMs, and effectively reflects diverse personalities. We evaluate nine LLMs on ToMATO and find that even GPT-4o mini lags behind human performance, especially in understanding false beliefs, and lacks robustness to various personality traits.