NiceWebRL: a Python library for human subject experiments with reinforcement learning environments

TL;DR

NiceWebRL is a Python library that transforms RL environments into online interfaces for human experiments, enabling comparisons between human and AI performance across various domains and supporting multi-agent collaboration research.

Contribution

It introduces a versatile Python tool that converts Jax-based RL environments into online interfaces for human subject experiments, facilitating new research in human-AI interaction.

Findings

Enabled testing of human-like RL models in grid world and Minecraft domains.

Developed a multi-agent RL algorithm that generalizes to human partners in Overcooked.

Studied LLM assistance in complex hierarchical tasks within XLand-Minigrid.

Abstract

We present NiceWebRL, a research tool that enables researchers to use machine reinforcement learning (RL) environments for online human subject experiments. NiceWebRL is a Python library that allows any Jax-based environment to be transformed into an online interface, supporting both single-agent and multi-agent environments. As such, NiceWebRL enables AI researchers to compare their algorithms to human performance, cognitive scientists to test ML algorithms as theories for human cognition, and multi-agent researchers to develop algorithms for human-AI collaboration. We showcase NiceWebRL with 3 case studies that demonstrate its potential to help develop Human-like AI, Human-compatible AI, and Human-assistive AI. In the first case study (Human-like AI), NiceWebRL enables the development of a novel RL model of cognition. Here, NiceWebRL facilitates testing this model against human participants in both a grid world and Craftax, a 2D Minecraft domain. In our second case study (Human-compatible AI), NiceWebRL enables the development of a novel multi-agent RL algorithm that can generalize to human partners in the Overcooked domain. Finally, in our third case study (Human-assistive AI), we show how NiceWebRL can allow researchers to study how an LLM can assist humans on complex tasks in XLand-Minigrid, an environment with millions of hierarchical tasks. The library is available at https://github.com/KempnerInstitute/nicewebrl.