Faster Deep Q-learning using Neural Episodic Control

TL;DR

This paper introduces NEC2DQN, a hybrid approach combining Neural Episodic Control with Deep Q-Networks to accelerate learning speed in deep reinforcement learning, demonstrated on Pong.

Contribution

It presents NEC2DQN, a novel method that enhances sample efficiency and learning speed by leveraging NEC at the start of training.

Findings

NEC2DQN learns faster than Double DQN in Pong.

NEC2DQN outperforms N-step DQN in early learning stages.

The approach improves sample efficiency in deep reinforcement learning.

Abstract

The research on deep reinforcement learning which estimates Q-value by deep learning has been attracted the interest of researchers recently. In deep reinforcement learning, it is important to efficiently learn the experiences that an agent has collected by exploring environment. We propose NEC2DQN that improves learning speed of a poor sample efficiency algorithm such as DQN by using good one such as NEC at the beginning of learning. We show it is able to learn faster than Double DQN or N-step DQN in the experiments of Pong.