Battlesnake Challenge: A Multi-agent Reinforcement Learning Playground with Human-in-the-loop

TL;DR

This paper introduces the Battlesnake Challenge, a multi-agent reinforcement learning framework with human-in-the-loop learning, enabling competitive agent training and evaluation in a multiplayer Snake game environment.

Contribution

It presents a flexible, agent-agnostic framework for multi-agent RL with human-in-the-loop, including both offline training and online competition modules.

Findings

HILL methods improve agent performance over non-HILL agents.

Reward manipulation heuristics yield the best online competition results.

Framework is validated with preliminary experiments showing effectiveness.

Abstract

We present the Battlesnake Challenge, a framework for multi-agent reinforcement learning with Human-In-the-Loop Learning (HILL). It is developed upon Battlesnake, a multiplayer extension of the traditional Snake game in which 2 or more snakes compete for the final survival. The Battlesnake Challenge consists of an offline module for model training and an online module for live competitions. We develop a simulated game environment for the offline multi-agent model training and identify a set of baseline heuristics that can be instilled to improve learning. Our framework is agent-agnostic and heuristics-agnostic such that researchers can design their own algorithms, train their models, and demonstrate in the online Battlesnake competition. We validate the framework and baseline heuristics with our preliminary experiments. Our results show that agents with the proposed HILL methods consistently outperform agents without HILL. Besides, heuristics of reward manipulation had the best performance in the online competition. We open source our framework at https://github.com/awslabs/sagemaker-battlesnake-ai.