I Know How: Combining Prior Policies to Solve New Tasks

TL;DR

The paper introduces I Know How (IKH), a modular framework for multi-task reinforcement learning that leverages prior knowledge to improve adaptation and efficiency in dynamic environments, demonstrated through a simulated driving task.

Contribution

The paper proposes a novel formalization for multi-task RL that emphasizes modularity and compositionality, unifying various approaches under a common framework.

Findings

IKH outperforms state-of-the-art methods in simulated driving tasks.

Modularity enhances learning efficiency and adaptability.

Framework reduces catastrophic forgetting.

Abstract

Multi-Task Reinforcement Learning aims at developing agents that are able to continually evolve and adapt to new scenarios. However, this goal is challenging to achieve due to the phenomenon of catastrophic forgetting and the high demand of computational resources. Learning from scratch for each new task is not a viable or sustainable option, and thus agents should be able to collect and exploit prior knowledge while facing new problems. While several methodologies have attempted to address the problem from different perspectives, they lack a common structure. In this work, we propose a new framework, I Know How (IKH), which provides a common formalization. Our methodology focuses on modularity and compositionality of knowledge in order to achieve and enhance agent's ability to learn and adapt efficiently to dynamic environments. To support our framework definition, we present a simple application of it in a simulated driving environment and compare its performance with that of state-of-the-art approaches.