Meta Reinforcement Learning with Latent Variable Gaussian Processes

TL;DR

This paper introduces a hierarchical latent variable model for meta reinforcement learning that automatically infers task relationships from data, enabling efficient generalization to new tasks with minimal data and reducing interaction time by up to 60%.

Contribution

It presents a novel latent variable framework for meta learning in reinforcement learning that eliminates the need for human-defined task relationships.

Findings

Achieves up to 60% reduction in interaction time for new tasks.

Effectively generalizes to unseen tasks using minimal data.

Automatically infers task relationships without human input.

Abstract

Learning from small data sets is critical in many practical applications where data collection is time consuming or expensive, e.g., robotics, animal experiments or drug design. Meta learning is one way to increase the data efficiency of learning algorithms by generalizing learned concepts from a set of training tasks to unseen, but related, tasks. Often, this relationship between tasks is hard coded or relies in some other way on human expertise. In this paper, we frame meta learning as a hierarchical latent variable model and infer the relationship between tasks automatically from data. We apply our framework in a model-based reinforcement learning setting and show that our meta-learning model effectively generalizes to novel tasks by identifying how new tasks relate to prior ones from minimal data. This results in up to a 60% reduction in the average interaction time needed to solve tasks compared to strong baselines.