XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning

TL;DR

XLand-100B is a large-scale, challenging dataset designed to advance in-context reinforcement learning research by providing extensive task histories, covering 100 billion transitions, and highlighting current baseline limitations.

Contribution

The paper introduces XLand-100B, a comprehensive large-scale dataset for in-context reinforcement learning, along with utilities and benchmarks to foster further research.

Findings

Baseline algorithms struggle to generalize to new tasks.

The dataset contains 100 billion transitions across 30,000 tasks.

Collecting the dataset required 50,000 GPU hours.

Abstract

Following the success of the in-context learning paradigm in large-scale language and computer vision models, the recently emerging field of in-context reinforcement learning is experiencing a rapid growth. However, its development has been held back by the lack of challenging benchmarks, as all the experiments have been carried out in simple environments and on small-scale datasets. We present XLand-100B, a large-scale dataset for in-context reinforcement learning based on the XLand-MiniGrid environment, as a first step to alleviate this problem. It contains complete learning histories for nearly $30,000$ different tasks, covering $100$B transitions and 2.5B episodes. It took 50,000 GPU hours to collect the dataset, which is beyond the reach of most academic labs. Along with the dataset, we provide the utilities to reproduce or expand it even further. We also benchmark common in-context RL baselines and show that they struggle to generalize to novel and diverse tasks. With this substantial effort, we aim to democratize research in the rapidly growing field of in-context reinforcement learning and provide a solid foundation for further scaling.