Model-Free Episodic Control

TL;DR

This paper introduces a model of hippocampal episodic control that enables rapid learning in reinforcement learning tasks, outperforming traditional deep RL algorithms in speed and sometimes in reward.

Contribution

It presents a simple episodic control model inspired by the hippocampus, demonstrating faster and sometimes more effective learning in complex decision tasks.

Findings

Faster convergence than deep RL algorithms.

Higher rewards in challenging domains.

Effective episodic memory-based learning.

Abstract

State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and its capacity for episodic memory. Here we investigate whether a simple model of hippocampal episodic control can learn to solve difficult sequential decision-making tasks. We demonstrate that it not only attains a highly rewarding strategy significantly faster than state-of-the-art deep reinforcement learning algorithms, but also achieves a higher overall reward on some of the more challenging domains.