Reasoning Promotes Robustness in Theory of Mind Tasks

TL;DR

This paper investigates how reasoning-focused large language models demonstrate increased robustness in Theory of Mind tasks, suggesting improvements are due to better problem-solving stability rather than new cognitive abilities.

Contribution

It provides evidence that reasoning training enhances robustness in ToM tasks, clarifying that gains are likely due to improved solution stability rather than novel reasoning capabilities.

Findings

Reasoning models show increased robustness to prompt variations.

Gains are attributed to better solution stability, not new ToM reasoning.

Analysis challenges the interpretation of ToM performance as social cognition.

Abstract

Large language models (LLMs) have recently shown strong performance on Theory of Mind (ToM) tests, prompting debate about the nature and true performance of the underlying capabilities. At the same time, reasoning-oriented LLMs trained via reinforcement learning with verifiable rewards (RLVR) have achieved notable improvements across a range of benchmarks. This paper examines the behavior of such reasoning models in ToM tasks, using novel adaptations of machine psychological experiments and results from established benchmarks. We observe that reasoning models consistently exhibit increased robustness to prompt variations and task perturbations. Our analysis indicates that the observed gains are more plausibly attributed to increased robustness in finding the correct solution, rather than to fundamentally new forms of ToM reasoning. We discuss the implications of this interpretation for evaluating social-cognitive behavior in LLMs.