Learning to Compose Skills
Himanshu Sahni, Saurabh Kumar, Farhan Tejani, Charles Isbell

TL;DR
This paper introduces a differentiable framework for composing simple policies into complex hierarchical skills, enabling rapid learning and transfer to new task combinations in reinforcement learning environments.
Contribution
It proposes a novel recursive skill composition architecture that generalizes to unseen skill combinations with zero-shot transfer capabilities.
Findings
Successfully builds complex skills from simple ones
Enables zero-shot generalization to new skill combinations
Demonstrates effectiveness in multi-task collect and evade environment
Abstract
We present a differentiable framework capable of learning a wide variety of compositions of simple policies that we call skills. By recursively composing skills with themselves, we can create hierarchies that display complex behavior. Skill networks are trained to generate skill-state embeddings that are provided as inputs to a trainable composition function, which in turn outputs a policy for the overall task. Our experiments on an environment consisting of multiple collect and evade tasks show that this architecture is able to quickly build complex skills from simpler ones. Furthermore, the learned composition function displays some transfer to unseen combinations of skills, allowing for zero-shot generalizations.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Topic Modeling · Machine Learning and Algorithms
