Been There, Done That: Meta-Learning with Episodic Recall

TL;DR

This paper introduces a meta-learning architecture combining LSTM and neural episodic memory to enable agents to remember and efficiently handle recurring tasks in open-ended environments.

Contribution

It presents a novel episodic LSTM architecture and a formalism for open-ended, repetitive environments, advancing meta-learning in natural, reoccurring task settings.

Findings

Episodic LSTM improves memory retention for recurring tasks.

Agents outperform standard meta-learning models in environments with reoccurring tasks.

The approach generalizes across diverse meta-learning scenarios.

Abstract

Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur - as they do in natural environments - metalearning agents must explore again instead of immediately exploiting previously discovered solutions. We propose a formalism for generating open-ended yet repetitious environments, then develop a meta-learning architecture for solving these environments. This architecture melds the standard LSTM working memory with a differentiable neural episodic memory. We explore the capabilities of agents with this episodic LSTM in five meta-learning environments with reoccurring tasks, ranging from bandits to navigation and stochastic sequential decision problems.