SHARPIE: A Modular Framework for Reinforcement Learning and Human-AI Interaction Experiments

TL;DR

SHARPIE is a flexible, modular platform designed to facilitate research on human-AI interaction in reinforcement learning, supporting experiments with diverse interaction paradigms and scalable deployment.

Contribution

It introduces a comprehensive, modular framework that standardizes and streamlines experiments involving human-AI interactions in reinforcement learning.

Findings

Supports a wide range of human-AI interaction experiments

Enables scalable deployment and recruitment for studies

Standardizes interfaces for human-RL interactions

Abstract

Reinforcement learning (RL) offers a general approach for modeling and training AI agents, including human-AI interaction scenarios. In this paper, we propose SHARPIE (Shared Human-AI Reinforcement Learning Platform for Interactive Experiments) to address the need for a generic framework to support experiments with RL agents and humans. Its modular design consists of a versatile wrapper for RL environments and algorithm libraries, a participant-facing web interface, logging utilities, deployment on popular cloud and participant recruitment platforms. It empowers researchers to study a wide variety of research questions related to the interaction between humans and RL agents, including those related to interactive reward specification and learning, learning from human feedback, action delegation, preference elicitation, user-modeling, and human-AI teaming. The platform is based on a generic interface for human-RL interactions that aims to standardize the field of study on RL in human contexts.