Lightning Does Not Strike Twice: Robust MDPs with Coupled Uncertainty

TL;DR

This paper introduces a novel approach to robust Markov decision processes by modeling coupled uncertainties with a bounded deviation concept, leading to less conservative solutions and practical algorithms for optimal control.

Contribution

It presents a new coupled uncertainty model called 'Lightning Does not Strike Twice' and develops tractable algorithms for optimal policies under this model.

Findings

Probabilistic guarantees for real-life applicability

Less conservative solutions compared to uncoupled models

Efficient algorithms for coupled uncertainty control

Abstract

We consider Markov decision processes under parameter uncertainty. Previous studies all restrict to the case that uncertainties among different states are uncoupled, which leads to conservative solutions. In contrast, we introduce an intuitive concept, termed "Lightning Does not Strike Twice," to model coupled uncertain parameters. Specifically, we require that the system can deviate from its nominal parameters only a bounded number of times. We give probabilistic guarantees indicating that this model represents real life situations and devise tractable algorithms for computing optimal control policies using this concept.