Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models

TL;DR

This paper critically evaluates large language models' neural theory of mind abilities, revealing they are fragile, heuristic-based, and not reliably robust across diverse tasks and adversarial challenges.

Contribution

It provides a comprehensive, stress-tested evaluation of LLMs' neural theory of mind, highlighting their limitations and the pitfalls of anecdotal and limited benchmark assessments.

Findings

LLMs show some N-ToM abilities but lack robustness

Models rely on shallow heuristics, failing adversarial tests

Anecdotal evidence is insufficient for assessing N-ToM capabilities

Abstract

The escalating debate on AI's capabilities warrants developing reliable metrics to assess machine "intelligence". Recently, many anecdotal examples were used to suggest that newer large language models (LLMs) like ChatGPT and GPT-4 exhibit Neural Theory-of-Mind (N-ToM); however, prior work reached conflicting conclusions regarding those abilities. We investigate the extent of LLMs' N-ToM through an extensive evaluation on 6 tasks and find that while LLMs exhibit certain N-ToM abilities, this behavior is far from being robust. We further examine the factors impacting performance on N-ToM tasks and discover that LLMs struggle with adversarial examples, indicating reliance on shallow heuristics rather than robust ToM abilities. We caution against drawing conclusions from anecdotal examples, limited benchmark testing, and using human-designed psychological tests to evaluate models.