Learning to Engage with Interactive Systems: A Field Study on Deep Reinforcement Learning in a Public Museum

TL;DR

This study develops a reinforcement learning approach to enable physical agents to generate engaging group interactions in a museum setting, using engagement estimation as a reward to improve robot responsiveness and likability.

Contribution

It introduces a novel method for estimating engagement in group interactions and applies reinforcement learning to develop adaptive, engaging behaviors in physical agents within real-world environments.

Findings

Adaptive behaviors increase engagement levels.

Reinforcement learning outperforms scripted behaviors.

Participants preferred adaptive interactions.

Abstract

Physical agents that can autonomously generate engaging, life-like behaviour will lead to more responsive and interesting robots and other autonomous systems. Although many advances have been made for one-to-one interactions in well controlled settings, future physical agents should be capable of interacting with humans in natural settings, including group interaction. In order to generate engaging behaviours, the autonomous system must first be able to estimate its human partners' engagement level. In this paper, we propose an approach for estimating engagement during group interaction by simultaneously taking into account active and passive interaction, i.e. occupancy, and use the measure as the reward signal within a reinforcement learning framework to learn engaging interactive behaviours. The proposed approach is implemented in an interactive sculptural system in a museum setting. We compare the learning system to a baseline using pre-scripted interactive behaviours. Analysis based on sensory data and survey data shows that adaptable behaviours within an expert-designed action space can achieve higher engagement and likeability.