Violation of Expectation via Metacognitive Prompting Reduces Theory of Mind Prediction Error in Large Language Models

TL;DR

This paper introduces a metacognitive prompting framework using Violation of Expectation to improve Theory of Mind predictions in Large Language Models, reducing errors and enabling better user modeling.

Contribution

It presents a novel application of developmental psychology concepts to LLMs, enhancing their ability to model user mental states through a new VoE-based method.

Findings

LLMs can learn about users by storing and retrieving expectation-violation facts.

Metacognitive prompting reduces prediction errors in user modeling.

The approach echoes human learning theories.

Abstract

Recent research shows that Large Language Models (LLMs) exhibit a compelling level of proficiency in Theory of Mind (ToM) tasks. This ability to impute unobservable mental states to others is vital to human social cognition and may prove equally important in principal-agent relations between individual humans and Artificial Intelligences (AIs). In this paper, we explore how a mechanism studied in developmental psychology known as Violation of Expectation (VoE) can be implemented to reduce errors in LLM prediction about users by leveraging emergent ToM affordances. And we introduce a \textit{metacognitive prompting} framework to apply VoE in the context of an AI tutor. By storing and retrieving facts derived in cases where LLM expectation about the user was violated, we find that LLMs are able to learn about users in ways that echo theories of human learning. Finally, we discuss latent hazards and augmentative opportunities associated with modeling user psychology and propose ways to mitigate risk along with possible directions for future inquiry.