Learning to Plan & Schedule with Reinforcement-Learned Bimanual Robot Skills

TL;DR

This paper presents a hierarchical reinforcement learning framework for complex bimanual robot manipulation, integrating skill planning and scheduling to improve success rates and coordination over traditional methods.

Contribution

Introduces a hierarchical framework combining RL-trained primitive skills with a Transformer-based high-level planner for integrated skill scheduling and execution.

Findings

Achieves higher success rates on contact-rich tasks.

Produces more efficient and coordinated behaviors.

Outperforms end-to-end RL approaches.

Abstract

Long-horizon contact-rich bimanual manipulation presents a significant challenge, requiring complex coordination involving a mixture of parallel execution and sequential collaboration between arms. In this paper, we introduce a hierarchical framework that frames this challenge as an integrated skill planning & scheduling problem, going beyond purely sequential decision-making to support simultaneous skill invocation. Our approach is built upon a library of single-arm and bimanual primitive skills, each trained using Reinforcement Learning (RL) in GPU-accelerated simulation. We then train a Transformer-based planner on a dataset of skill compositions to act as a high-level scheduler, simultaneously predicting the discrete schedule of skills as well as their continuous parameters. We demonstrate that our method achieves higher success rates on complex, contact-rich tasks than end-to-end RL approaches and produces more efficient, coordinated behaviors than traditional sequential-only planners.