Model-Based Bayesian Reinforcement Learning in Large Structured Domains

TL;DR

This paper introduces a scalable Bayesian reinforcement learning framework that combines factored representations with online planning to efficiently learn and plan in large structured domains.

Contribution

It presents a novel Bayesian approach that jointly learns the structure and parameters of dynamical systems while planning actions in large domains.

Findings

Enhanced scalability of Bayesian RL methods

Effective joint structure and parameter learning

Near-optimal planning in complex environments

Abstract

Model-based Bayesian reinforcement learning has generated significant interest in the AI community as it provides an elegant solution to the optimal exploration-exploitation tradeoff in classical reinforcement learning. Unfortunately, the applicability of this type of approach has been limited to small domains due to the high complexity of reasoning about the joint posterior over model parameters. In this paper, we consider the use of factored representations combined with online planning techniques, to improve scalability of these methods. The main contribution of this paper is a Bayesian framework for learning the structure and parameters of a dynamical system, while also simultaneously planning a (near-)optimal sequence of actions.