Interpretable Dynamics Models for Data-Efficient Reinforcement Learning

TL;DR

This paper introduces a Bayesian approach to model-based reinforcement learning that leverages expert knowledge for structured transition models, enhancing data efficiency and interpretability compared to existing methods.

Contribution

It presents a novel variational inference scheme for structured transition models that incorporate expert knowledge, improving data efficiency and interpretability in reinforcement learning.

Findings

Outperforms NFQ on a heteroskedastic and bimodal benchmark

Provides human-interpretable insights into dynamics

Increases data efficiency in learning

Abstract

In this paper, we present a Bayesian view on model-based reinforcement learning. We use expert knowledge to impose structure on the transition model and present an efficient learning scheme based on variational inference. This scheme is applied to a heteroskedastic and bimodal benchmark problem on which we compare our results to NFQ and show how our approach yields human-interpretable insight about the underlying dynamics while also increasing data-efficiency.