Shields to Guarantee Probabilistic Safety in MDPs

TL;DR

This paper extends classical shielding techniques to probabilistic safety in MDPs, providing formal frameworks, weaker guarantees, and practical shield constructions with empirical validation.

Contribution

It introduces a formal framework for probabilistic shielding, demonstrates the limitations of strong guarantees, and proposes new offline and online shield constructions.

Findings

Weakening safety guarantees enables probabilistic shielding.

Offline and online shields ensure strong safety guarantees.

Empirical evaluation shows practical feasibility and advantages.

Abstract

Shielding is a prominent model-based technique to ensure safety of autonomous agents. Classical shielding aims to ensure that nothing bad ever happens and comes with strong guarantees about safety and maximal permissiveness. However, shielding systems for probabilistic safety, where something bad is allowed to happen with an acceptable probability, has proven to be more intricate. This paper presents a formal framework that conservatively extends classical shields to probabilistic safety. In this framework, we (i) demonstrate the impossibility of preserving the strong guarantees on safety and permissiveness, (ii) provide natural shields with weaker guarantees, and (iii) introduce offline and online shield constructions ensuring strong safety guarantees. The empirical evaluation highlights the practical advantages of the new shields, as well as their computational feasibility.