Learning Parameterized Skills

TL;DR

This paper presents a method for learning parameterized skills in reinforcement learning by modeling the manifold of skill policies across task variations, enabling the robot to adapt to different target locations.

Contribution

The paper introduces a novel approach to construct parameterized skills by estimating the manifold of policies and applying non-linear regression within its charts.

Findings

Successfully learned skills for a robotic arm to throw darts at varying targets.

Demonstrated the ability to model the policy manifold across task parameters.

Achieved accurate policy predictions for new task parameters.

Abstract

We introduce a method for constructing skills capable of solving tasks drawn from a distribution of parameterized reinforcement learning problems. The method draws example tasks from a distribution of interest and uses the corresponding learned policies to estimate the topology of the lower-dimensional piecewise-smooth manifold on which the skill policies lie. This manifold models how policy parameters change as task parameters vary. The method identifies the number of charts that compose the manifold and then applies non-linear regression in each chart to construct a parameterized skill by predicting policy parameters from task parameters. We evaluate our method on an underactuated simulated robotic arm tasked with learning to accurately throw darts at a parameterized target location.