Exploration for Multi-task Reinforcement Learning with Deep Generative Models

TL;DR

This paper introduces a novel exploration method for multi-task reinforcement learning using deep generative models and energy-based models to adaptively identify underlying MDPs across multiple tasks.

Contribution

It proposes a new approach combining deep generative models and energy models for effective exploration in multi-task reinforcement learning, addressing limitations of existing methods.

Findings

Effective exploration in multi-task settings demonstrated

Energy model provides resilient exploration signals

Method outperforms traditional single-task exploration algorithms

Abstract

Exploration in multi-task reinforcement learning is critical in training agents to deduce the underlying MDP. Many of the existing exploration frameworks such as $E^3$, $R_{max}$, Thompson sampling assume a single stationary MDP and are not suitable for system identification in the multi-task setting. We present a novel method to facilitate exploration in multi-task reinforcement learning using deep generative models. We supplement our method with a low dimensional energy model to learn the underlying MDP distribution and provide a resilient and adaptive exploration signal to the agent. We evaluate our method on a new set of environments and provide intuitive interpretation of our results.