Accuracy-based Curriculum Learning in Deep Reinforcement Learning

TL;DR

This paper introduces accuracy-based curriculum learning in deep reinforcement learning, where adaptive accuracy requirements guide training, leading to improved efficiency over random sampling.

Contribution

It proposes a novel adaptive curriculum learning method based on accuracy requirements and competence progress, enhancing training efficiency in deep RL.

Findings

Adaptive accuracy requirements improve learning efficiency.

Curriculum automatically increases difficulty over time.

Method outperforms random sampling in experiments.

Abstract

In this paper, we investigate a new form of automated curriculum learning based on adaptive selection of accuracy requirements, called accuracy-based curriculum learning. Using a reinforcement learning agent based on the Deep Deterministic Policy Gradient algorithm and addressing the Reacher environment, we first show that an agent trained with various accuracy requirements sampled randomly learns more efficiently than when asked to be very accurate at all times. Then we show that adaptive selection of accuracy requirements, based on a local measure of competence progress, automatically generates a curriculum where difficulty progressively increases, resulting in a better learning efficiency than sampling randomly.