Learning to Compose Skills

TL;DR

This paper introduces a differentiable framework for composing simple policies into complex hierarchical skills, enabling rapid learning and transfer to new task combinations in reinforcement learning environments.

Contribution

It proposes a novel recursive skill composition architecture that generalizes to unseen skill combinations with zero-shot transfer capabilities.

Findings

Successfully builds complex skills from simple ones

Enables zero-shot generalization to new skill combinations

Demonstrates effectiveness in multi-task collect and evade environment

Abstract

We present a differentiable framework capable of learning a wide variety of compositions of simple policies that we call skills. By recursively composing skills with themselves, we can create hierarchies that display complex behavior. Skill networks are trained to generate skill-state embeddings that are provided as inputs to a trainable composition function, which in turn outputs a policy for the overall task. Our experiments on an environment consisting of multiple collect and evade tasks show that this architecture is able to quickly build complex skills from simpler ones. Furthermore, the learned composition function displays some transfer to unseen combinations of skills, allowing for zero-shot generalizations.