Enhancing Robot Assistive Behaviour with Reinforcement Learning and Theory of Mind

TL;DR

This study explores how social robots with Theory of Mind (ToM) abilities influence user performance and perception, showing that ToM-equipped robots improve interaction outcomes in real-world human-robot collaboration.

Contribution

The paper introduces a two-layer architecture combining reinforcement learning and ToM for adaptive robot assistance, and provides empirical evidence of its positive impact on users.

Findings

Participants with ToM robots performed better.

ToM robots were more accepted by users.

Users perceived ToM robots as more adaptive and predictive.

Abstract

The adaptation to users' preferences and the ability to infer and interpret humans' beliefs and intents, which is known as the Theory of Mind (ToM), are two crucial aspects for achieving effective human-robot collaboration. Despite its importance, very few studies have investigated the impact of adaptive robots with ToM abilities. In this work, we present an exploratory comparative study to investigate how social robots equipped with ToM abilities impact users' performance and perception. We design a two-layer architecture. The Q-learning agent on the first layer learns the robot's higher-level behaviour. On the second layer, a heuristic-based ToM infers the user's intended strategy and is responsible for implementing the robot's assistance, as well as providing the motivation behind its choice. We conducted a user study in a real-world setting, involving 56 participants who interacted with either an adaptive robot capable of ToM, or with a robot lacking such abilities. Our findings suggest that participants in the ToM condition performed better, accepted the robot's assistance more often, and perceived its ability to adapt, predict and recognise their intents to a higher degree. Our preliminary insights could inform future research and pave the way for designing more complex computation architectures for adaptive behaviour with ToM capabilities.