Neural Episodic Control

TL;DR

Neural Episodic Control is a reinforcement learning approach that rapidly learns from new experiences using a semi-tabular value function representation, achieving faster learning than existing methods.

Contribution

Introduces Neural Episodic Control, a novel deep RL agent combining semi-tabular value estimates with experience buffers for rapid learning.

Findings

Learns significantly faster than state-of-the-art deep RL agents

Effective across diverse environments

Utilizes a semi-tabular value function representation

Abstract

Deep reinforcement learning methods attain super-human performance in a wide range of environments. Such methods are grossly inefficient, often taking orders of magnitudes more data than humans to achieve reasonable performance. We propose Neural Episodic Control: a deep reinforcement learning agent that is able to rapidly assimilate new experiences and act upon them. Our agent uses a semi-tabular representation of the value function: a buffer of past experience containing slowly changing state representations and rapidly updated estimates of the value function. We show across a wide range of environments that our agent learns significantly faster than other state-of-the-art, general purpose deep reinforcement learning agents.