# Meta Reinforcement Learning with Distribution of Exploration Parameters Learned by Evolution Strategies

## TL;DR

This paper introduces a meta-learning approach in reinforcement learning that combines evolution strategies with deterministic policy gradients, achieving scalable and sample-efficient adaptation in high-dimensional control tasks.

## Contribution

It presents a novel meta-learning method using evolution strategies and deterministic policy gradients, improving sample efficiency and scalability in reinforcement learning.

## Key findings

- Achieves competitive results in MuJoCo control tasks.
- Performs better in multi-step adaptation scenarios.
- Maintains scalability with parallelizable evolution strategies.

## Abstract

In this paper, we propose a novel meta-learning method in a reinforcement learning setting, based on evolution strategies (ES), exploration in parameter space and deterministic policy gradients. ES methods are easy to parallelize, which is desirable for modern training architectures; however, such methods typically require a huge number of samples for effective training. We use deterministic policy gradients during adaptation and other techniques to compensate for the sample-efficiency problem while maintaining the inherent scalability of ES methods. We demonstrate that our method achieves good results compared to gradient-based meta-learning in high-dimensional control tasks in the MuJoCo simulator. In addition, because of gradient-free methods in the meta-training phase, which do not need information about gradients and policies in adaptation training, we predict and confirm our algorithm performs better in tasks that need multi-step adaptation.