# Learning Efficient and Effective Exploration Policies with Counterfactual Meta Policy

## TL;DR

This paper introduces a meta-learning approach to develop exploration policies in reinforcement learning, utilizing a counterfactual utility metric to improve efficiency and effectiveness across different environments.

## Contribution

It proposes a novel end-to-end meta-learning algorithm for exploration policy learning based on a counterfactual utility metric, addressing task-specific limitations.

## Key findings

- Achieves superior performance in high-dimensional control tasks
- Demonstrates improved exploration efficiency over previous methods
- Validates effectiveness across MuJoCo simulation environments

## Abstract

A fundamental issue in reinforcement learning algorithms is the balance between exploration of the environment and exploitation of information already obtained by the agent. Especially, exploration has played a critical role for both efficiency and efficacy of the learning process. However, Existing works for exploration involve task-agnostic design, that is performing well in one environment, but be ill-suited to another. To the purpose of learning an effective and efficient exploration policy in an automated manner. We formalized a feasible metric for measuring the utility of exploration based on counterfactual ideology. Based on that, We proposed an end-to-end algorithm to learn exploration policy by meta-learning. We demonstrate that our method achieves good results compared to previous works in the high-dimensional control tasks in MuJoCo simulator.