Progressive Neural Networks

TL;DR

Progressive neural networks are designed to learn sequences of tasks without forgetting, effectively transfer knowledge, and outperform traditional methods in reinforcement learning environments like Atari and 3D maze games.

Contribution

This paper introduces progressive neural networks that prevent catastrophic forgetting and enable transfer learning through lateral connections, advancing continual learning capabilities.

Findings

Outperform baseline methods in reinforcement learning tasks

Transfer occurs at both sensory and control layers

Networks are immune to forgetting during sequential learning

Abstract

Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence. The progressive networks approach represents a step forward in this direction: they are immune to forgetting and can leverage prior knowledge via lateral connections to previously learned features. We evaluate this architecture extensively on a wide variety of reinforcement learning tasks (Atari and 3D maze games), and show that it outperforms common baselines based on pretraining and finetuning. Using a novel sensitivity measure, we demonstrate that transfer occurs at both low-level sensory and high-level control layers of the learned policy.