Achieving Goals using Reward Shaping and Curriculum Learning

TL;DR

This paper presents a method combining curriculum learning, reward shaping, and parallel environments to efficiently train agents for complex goal-oriented robotic tasks, specifically stacking cubes, by breaking down the task into manageable sub-goals.

Contribution

It introduces two curriculum learning settings for goal-conditioned tasks, enabling the learning of complex tasks without extensive architectural changes.

Findings

Effective task decomposition into sub-goals

Successful application to cube stacking with varied object shapes

Enhanced learning efficiency through parallel environments

Abstract

Real-time control for robotics is a popular research area in the reinforcement learning community. Through the use of techniques such as reward shaping, researchers have managed to train online agents across a multitude of domains. Despite these advances, solving goal-oriented tasks still requires complex architectural changes or hard constraints to be placed on the problem. In this article, we solve the problem of stacking multiple cubes by combining curriculum learning, reward shaping, and a high number of efficiently parallelized environments. We introduce two curriculum learning settings that allow us to separate the complex task into sequential sub-goals, hence enabling the learning of a problem that may otherwise be too difficult. We focus on discussing the challenges encountered while implementing them in a goal-conditioned environment. Finally, we extend the best configuration identified on a higher complexity environment with differently shaped objects.