Grounded Curriculum Learning

TL;DR

Grounded Curriculum Learning (GCL) improves robotics reinforcement learning by aligning simulated training tasks with real-world distributions, leading to better navigation success rates and more efficient learning.

Contribution

GCL introduces a method to adaptively align simulation task distributions with real-world data, addressing a key mismatch in curriculum learning for robotics RL.

Findings

GCL achieved 6.8% higher success rate than state-of-the-art CL methods.

GCL outperformed human-designed curricula by 6.5% in success rate.

GCL enhances learning efficiency and navigation performance in complex tasks.

Abstract

The high cost of real-world data for robotics Reinforcement Learning (RL) leads to the wide usage of simulators. Despite extensive work on building better dynamics models for simulators to match with the real world, there is another, often-overlooked mismatch between simulations and the real world, namely the distribution of available training tasks. Such a mismatch is further exacerbated by existing curriculum learning techniques, which automatically vary the simulation task distribution without considering its relevance to the real world. Considering these challenges, we posit that curriculum learning for robotics RL needs to be grounded in real-world task distributions. To this end, we propose Grounded Curriculum Learning (GCL), which aligns the simulated task distribution in the curriculum with the real world, as well as explicitly considers what tasks have been given to the robot and how the robot has performed in the past. We validate GCL using the BARN dataset on complex navigation tasks, achieving a 6.8% and 6.5% higher success rate compared to a state-of-the-art CL method and a curriculum designed by human experts, respectively. These results show that GCL can enhance learning efficiency and navigation performance by grounding the simulation task distribution in the real world within an adaptive curriculum.