Combine PPO with NES to Improve Exploration

TL;DR

This paper proposes combining neural evolution strategy (NES) with proximal policy optimization (PPO) through parameter transfer and parameter space noise to enhance exploration in reinforcement learning.

Contribution

It introduces two novel methods for integrating NES with PPO, improving exploration capabilities in RL tasks.

Findings

PPO benefits from parameter transfer and space noise methods.

Enhanced exploration leads to better performance in discrete and continuous environments.

Experimental results show improved learning efficiency.

Abstract

We introduce two approaches for combining neural evolution strategy (NES) and proximal policy optimization (PPO): parameter transfer and parameter space noise. Parameter transfer is a PPO agent with parameters transferred from a NES agent. Parameter space noise is to directly add noise to the PPO agent`s parameters. We demonstrate that PPO could benefit from both methods through experimental comparison on discrete action environments as well as continuous control tasks