What Do You Think I Think? Accounting for Human Beliefs Using Second-Order Theory of Mind

TL;DR

This paper introduces a second-order Theory of Mind framework enabling agents to model and adapt to human beliefs and biases, improving interaction quality through tailored feedback.

Contribution

It develops a ToM-2 model based on I-POMDPs to detect and account for human cognitive biases during interactions, enhancing feedback effectiveness.

Findings

ToM-2 model improves informativeness of agent feedback.

Participants found ToM-2 feedback more useful.

Significant reduction in belief discrepancies during interactions.

Abstract

Discrepancies between an agent's actual knowledge and what a person thinks the agent knows can hinder interactions. If an agent could detect such discrepancies, it could provide feedback to account for them and improve current and future interactions. Using the I-POMDP as a framework for a second-order Theory of Mind (ToM-2), this work endows an agent with the ability to model the evolution of a person's erroneous beliefs about an agent and the cognitive biases and heuristics (CBH) from which they arise. In doing so, the agent can detect when CBH might be at play during an interaction and adaptively generate feedback that accounts for them. An in-person user study shows how a ToM-2 learner can account for the effects of a teacher's CBH to significantly improve the informativeness of teacher actions, and subjective results suggest people find the ToM-2 learner's feedback more useful.