Certifiably Robust Policies for Uncertain Parametric Environments
Yannik Schnitzer, Alessandro Abate, David Parker

TL;DR
This paper introduces a data-driven framework for creating policies that are provably robust across uncertain environments by combining parametric MDPs with scenario optimization, providing high-confidence performance guarantees.
Contribution
It proposes a novel approach that integrates parametric MDPs with scenario optimization to produce robust policies with quantifiable risk and performance guarantees.
Findings
Produces tight performance bounds with high confidence
Effectively handles multiple layers of environmental uncertainty
Demonstrates the approach on various benchmark problems
Abstract
We present a data-driven approach for producing policies that are provably robust across unknown stochastic environments. Existing approaches can learn models of a single environment as an interval Markov decision processes (IMDP) and produce a robust policy with a probably approximately correct (PAC) guarantee on its performance. However these are unable to reason about the impact of environmental parameters underlying the uncertainty. We propose a framework based on parametric Markov decision processes (MDPs) with unknown distributions over parameters. We learn and analyse IMDPs for a set of unknown sample environments induced by parameters. The key challenge is then to produce meaningful performance guarantees that combine the two layers of uncertainty: (1) multiple environments induced by parameters with an unknown distribution; (2) unknown induced environments which are…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsProbabilistic and Robust Engineering Design
MethodsSparse Evolutionary Training
