Learning Task Agnostic Skills with Data-driven Guidance

TL;DR

This paper introduces a framework that guides unsupervised skill discovery in reinforcement learning towards expert-visited states using a learned state projection, resulting in more useful behaviors without task-specific rewards.

Contribution

It presents a novel data-driven guidance method for skill discovery that focuses on expert-visited states to improve the usefulness of learned behaviors.

Findings

Guided skill discovery produces more relevant behaviors.

The method enhances autonomy without task-specific rewards.

Effective in various RL tasks.

Abstract

To increase autonomy in reinforcement learning, agents need to learn useful behaviours without reliance on manually designed reward functions. To that end, skill discovery methods have been used to learn the intrinsic options available to an agent using task-agnostic objectives. However, without the guidance of task-specific rewards, emergent behaviours are generally useless due to the under-constrained problem of skill discovery in complex and high-dimensional spaces. This paper proposes a framework for guiding the skill discovery towards the subset of expert-visited states using a learned state projection. We apply our method in various reinforcement learning (RL) tasks and show that such a projection results in more useful behaviours.