Robust Meta-Reinforcement Learning with Curriculum-Based Task Sampling

TL;DR

This paper introduces RMRL-GTS, a curriculum-based task sampling method for meta-reinforcement learning that reduces meta-overfitting by adaptively selecting tasks based on scores and epochs.

Contribution

It proposes a novel task sampling strategy that improves robustness in meta-RL by controlling task difficulty and diversity during training.

Findings

RMRL-GTS reduces meta-overfitting in meta-RL.

Adaptive task sampling improves policy robustness.

The method balances sampling from easy and hard tasks effectively.

Abstract

Meta-reinforcement learning (meta-RL) acquires meta-policies that show good performance for tasks in a wide task distribution. However, conventional meta-RL, which learns meta-policies by randomly sampling tasks, has been reported to show meta-overfitting for certain tasks, especially for easy tasks where an agent can easily get high scores. To reduce effects of the meta-overfitting, we considered meta-RL with curriculum-based task sampling. Our method is Robust Meta Reinforcement Learning with Guided Task Sampling (RMRL-GTS), which is an effective method that restricts task sampling based on scores and epochs. We show that in order to achieve robust meta-RL, it is necessary not only to intensively sample tasks with poor scores, but also to restrict and expand the task regions of the tasks to be sampled.