Adaptive Skills, Adaptive Partitions (ASAP)

TL;DR

The ASAP framework enables simultaneous learning of skills and their application locations, facilitating scalable, lifelong learning and efficient adaptation to new tasks with less experience.

Contribution

This paper introduces the ASAP framework that learns both skills and their application regions, a novel approach for scalable and adaptable lifelong learning.

Findings

ASAP converges to a local optimum.

ASAP effectively reuses skills across tasks.

ASAP reduces experience needed for new tasks.

Abstract

We introduce the Adaptive Skills, Adaptive Partitions (ASAP) framework that (1) learns skills (i.e., temporally extended actions or options) as well as (2) where to apply them. We believe that both (1) and (2) are necessary for a truly general skill learning framework, which is a key building block needed to scale up to lifelong learning agents. The ASAP framework can also solve related new tasks simply by adapting where it applies its existing learned skills. We prove that ASAP converges to a local optimum under natural conditions. Finally, our experimental results, which include a RoboCup domain, demonstrate the ability of ASAP to learn where to reuse skills as well as solve multiple tasks with considerably less experience than solving each task from scratch.