The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design

TL;DR

The paper introduces the Overcooked Generalisation Challenge (OGC), a new benchmark for testing reinforcement learning agents' ability to cooperate with unknown partners in diverse, unfamiliar environments, emphasizing the importance of generalisation.

Contribution

It extends the Overcooked-AI environment with richer environment design and dual curriculum support, providing a challenging benchmark for cooperative generalisation in RL.

Findings

Current algorithms struggle with partner and environment generalisation.

Existing methods fail to produce agents that adapt well to new layouts and partners.

OGC serves as a demanding testbed highlighting key future research directions.

Abstract

We introduce the Overcooked Generalisation Challenge (OGC) - a new benchmark for evaluating reinforcement learning (RL) agents on their ability to cooperate with unknown partners in unfamiliar environments. Existing work typically evaluated cooperative RL only in their training environment or with their training partners, thus seriously limiting our ability to understand agents' generalisation capacity - an essential requirement for future collaboration with humans. The OGC extends Overcooked-AI to support dual curriculum design (DCD). It is fully GPU-accelerated, open-source, and integrated into the minimax DCD benchmark suite. Compared to prior DCD benchmarks, where designers manipulate only minimal elements of the environment, OGC introduces a significantly richer design space: full kitchen layouts with multiple objects that require the designer to account for interaction dynamics between agents. We evaluate state-of-the-art DCD algorithms alongside scalable neural architectures and find that current methods fail to produce agents that generalise effectively to novel layouts and unfamiliar partners. Our results indicate that both agents and curriculum designers struggle with the joint challenge of partner and environment generalisation. These findings establish OGC as a demanding testbed for cooperative generalisation and highlight key directions for future research. We open-source our code.