Meta-Reinforcement Learning Using Model Parameters

TL;DR

This paper introduces RAMP, a meta-reinforcement learning approach that leverages environment dynamics models to enable efficient adaptation across multiple environments.

Contribution

RAMP is a novel method that uses learned environment models' parameters as context for policy adaptation in meta-reinforcement learning.

Findings

RAMP effectively captures environment dynamics.

It improves adaptation speed in new environments.

The approach demonstrates strong performance across tasks.

Abstract

In meta-reinforcement learning, an agent is trained in multiple different environments and attempts to learn a meta-policy that can efficiently adapt to a new environment. This paper presents RAMP, a Reinforcement learning Agent using Model Parameters that utilizes the idea that a neural network trained to predict environment dynamics encapsulates the environment information. RAMP is constructed in two phases: in the first phase, a multi-environment parameterized dynamic model is learned. In the second phase, the model parameters of the dynamic model are used as context for the multi-environment policy of the model-free reinforcement learning agent.