Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Robotics

TL;DR

This paper introduces Learning to Execute (L2E), a framework that combines reinforcement learning and planning to create universal, plan-conditioned policies in robotics, improving data efficiency and performance.

Contribution

The paper presents L2E, a novel method integrating RL and planning to learn universal policies conditioned on plans, addressing limitations of each approach individually.

Findings

L2E outperforms pure RL and planning methods in robotic manipulation tasks.

L2E demonstrates increased data efficiency and robustness.

Universal policies conditioned on plans improve task success rates.

Abstract

Applications of Reinforcement Learning (RL) in robotics are often limited by high data demand. On the other hand, approximate models are readily available in many robotics scenarios, making model-based approaches like planning a data-efficient alternative. Still, the performance of these methods suffers if the model is imprecise or wrong. In this sense, the respective strengths and weaknesses of RL and model-based planners are. In the present work, we investigate how both approaches can be integrated into one framework that combines their strengths. We introduce Learning to Execute (L2E), which leverages information contained in approximate plans to learn universal policies that are conditioned on plans. In our robotic manipulation experiments, L2E exhibits increased performance when compared to pure RL, pure planning, or baseline methods combining learning and planning.