# Combining Experience Replay with Exploration by Random Network Distillation

## TL;DR

This paper introduces a novel method combining experience replay with exploration using Random Network Distillation, leading to improved exploration efficiency and performance in challenging Atari games.

## Contribution

It proposes Prioritized Oversampled Experience Replay (POER), a new technique for selecting important experiences to enhance exploration and learning.

## Key findings

- POER improves sample efficiency in Atari games.
- The method outperforms PPO/RND in Montezuma's Revenge.
- Enhanced exploration leads to better agent performance.

## Abstract

Our work is a simple extension of the paper "Exploration by Random Network Distillation". More in detail, we show how to efficiently combine Intrinsic Rewards with Experience Replay in order to achieve more efficient and robust exploration (with respect to PPO/RND) and consequently better results in terms of agent performances and sample efficiency. We are able to do it by using a new technique named Prioritized Oversampled Experience Replay (POER), that has been built upon the definition of what is the important experience useful to replay. Finally, we evaluate our technique on the famous Atari game Montezuma's Revenge and some other hard exploration Atari games.