Rectangularity and duality of distributionally robust Markov Decision Processes

TL;DR

This paper explores different formulations of distributionally robust Markov Decision Processes, focusing on the role of rectangularity in ambiguity sets and clarifying connections between game and static approaches.

Contribution

It clarifies the relationships between game and static formulations of distributionally robust MDPs and examines the importance of rectangularity in ambiguity sets.

Findings

Rectangularity influences the equivalence of game and static formulations.

Connections between different formulations are clarified.

The role of ambiguity set structure in robust decision-making is elucidated.

Abstract

The main goal of this paper is to discuss several approaches to formulation of distributionally robust counterparts of Markov Decision Processes, where the transition kernels are not specified exactly but rather are assumed to be elements of the corresponding ambiguity sets. The intent is to clarify some connections between the game and static formulations of distributionally robust MDPs, and delineate the role of rectangularity associated with ambiguity sets in determining these connections.