Automating Curriculum Learning for Reinforcement Learning using a Skill-Based Bayesian Network

TL;DR

This paper presents SEBNs, a probabilistic model that automates curriculum generation in reinforcement learning by predicting skill success and selecting tasks to improve training efficiency across diverse environments.

Contribution

Introduction of SEBNs, a novel Bayesian network model that predicts policy performance and guides automated curriculum learning in reinforcement learning tasks.

Findings

SEBN-based curricula outperform baseline methods in various environments.

The approach reduces training time and improves policy success rates.

Effective across gridworld, continuous control, and robotics simulations.

Abstract

A major challenge for reinforcement learning is automatically generating curricula to reduce training time or improve performance in some target task. We introduce SEBNs (Skill-Environment Bayesian Networks) which model a probabilistic relationship between a set of skills, a set of goals that relate to the reward structure, and a set of environment features to predict policy performance on (possibly unseen) tasks. We develop an algorithm that uses the inferred estimates of agent success from SEBN to weigh the possible next tasks by expected improvement. We evaluate the benefit of the resulting curriculum on three environments: a discrete gridworld, continuous control, and simulated robotics. The results show that curricula constructed using SEBN frequently outperform other baselines.