Improving generalization in reinforcement learning through forked agents

TL;DR

This paper explores how different initialization techniques for agents in a reinforcement learning eco-system can improve adaptation speed and generalization across procedurally generated environments.

Contribution

It introduces novel initialization methods inspired by neural network initialization and transfer learning, and evaluates their impact on agent performance.

Findings

Initialization techniques significantly affect adaptation speed.

Transfer learning-inspired methods improve generalization.

Certain initializations outperform standard methods.

Abstract

An eco-system of agents each having their own policy with some, but limited, generalizability has proven to be a reliable approach to increase generalization across procedurally generated environments. In such an approach, new agents are regularly added to the eco-system when encountering a new environment that is outside of the scope of the eco-system. The speed of adaptation and general effectiveness of the eco-system approach highly depends on the initialization of new agents. In this paper we propose different initialization techniques, inspired from Deep Neural Network initialization and transfer learning, and study their impact.