Expert Selection in High-Dimensional Markov Decision Processes

TL;DR

This paper introduces a bandit-based framework for online expert selection in high-dimensional Markov decision processes, enabling rapid identification of the best expert policy with low regret in dynamic environments.

Contribution

It proposes a novel multi-armed bandit approach using an upper confidence bound algorithm tailored for expert selection in high-dimensional MDPs.

Findings

Effective rapid expert identification demonstrated

Low regret performance achieved in high-dimensional settings

Applicable to real-time decision-making scenarios

Abstract

In this work we present a multi-armed bandit framework for online expert selection in Markov decision processes and demonstrate its use in high-dimensional settings. Our method takes a set of candidate expert policies and switches between them to rapidly identify the best performing expert using a variant of the classical upper confidence bound algorithm, thus ensuring low regret in the overall performance of the system. This is useful in applications where several expert policies may be available, and one needs to be selected at run-time for the underlying environment.