EduGym: An Environment and Notebook Suite for Reinforcement Learning Education

TL;DR

EduGym offers specialized reinforcement learning environments and interactive notebooks designed to enhance educational understanding by illustrating core challenges and solutions, bridging the gap between theory and practice.

Contribution

It introduces EduGym, a novel suite of environments and notebooks tailored for reinforcement learning education, simplifying complex concepts for students and researchers.

Findings

86% of users found EduGym useful for RL education

Provides environments illustrating key RL challenges

Connects equations and code effectively

Abstract

Due to the empirical success of reinforcement learning, an increasing number of students study the subject. However, from our practical teaching experience, we see students entering the field (bachelor, master and early PhD) often struggle. On the one hand, textbooks and (online) lectures provide the fundamentals, but students find it hard to translate between equations and code. On the other hand, public codebases do provide practical examples, but the implemented algorithms tend to be complex, and the underlying test environments contain multiple reinforcement learning challenges at once. Although this is realistic from a research perspective, it often hinders educational conceptual understanding. To solve this issue we introduce EduGym, a set of educational reinforcement learning environments and associated interactive notebooks tailored for education. Each EduGym environment is specifically designed to illustrate a certain aspect/challenge of reinforcement learning (e.g., exploration, partial observability, stochasticity, etc.), while the associated interactive notebook explains the challenge and its possible solution approaches, connecting equations and code in a single document. An evaluation among RL students and researchers shows 86% of them think EduGym is a useful tool for reinforcement learning education. All notebooks are available from https://www.edugym.org/, while the full software package can be installed from https://github.com/RLG-Leiden/edugym.