Noisy Networks for Exploration

TL;DR

NoisyNet introduces learnable parametric noise into deep reinforcement learning agents, enhancing exploration efficiency and significantly improving performance across various Atari games compared to traditional exploration methods.

Contribution

The paper presents NoisyNet, a simple yet effective method of incorporating learnable noise into network weights to improve exploration in deep reinforcement learning.

Findings

NoisyNet outperforms traditional exploration heuristics in Atari games.

Replacing entropy and epsilon-greedy exploration with NoisyNet yields higher scores.

In some cases, NoisyNet achieves super-human performance.

Abstract

We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to aid efficient exploration. The parameters of the noise are learned with gradient descent along with the remaining network weights. NoisyNet is straightforward to implement and adds little computational overhead. We find that replacing the conventional exploration heuristics for A3C, DQN and dueling agents (entropy reward and $\epsilon$-greedy respectively) with NoisyNet yields substantially higher scores for a wide range of Atari games, in some cases advancing the agent from sub to super-human performance.