Generalised Entropy MDPs and Minimax Regret

TL;DR

This paper explores the use of minimax-Bayes policies in stochastic zero-sum games to address prior specification issues in Bayesian methods, extending bandit theory results for practical application.

Contribution

It introduces a generalized entropy framework for Markov Decision Processes and extends bandit theory to develop minimax-Bayes policies for worst-case prior scenarios.

Findings

Derived new minimax-Bayes policies for zero-sum games

Extended bandit theory results to generalised entropy MDPs

Discussed practical applicability of these policies

Abstract

Bayesian methods suffer from the problem of how to specify prior beliefs. One interesting idea is to consider worst-case priors. This requires solving a stochastic zero-sum game. In this paper, we extend well-known results from bandit theory in order to discover minimax-Bayes policies and discuss when they are practical.