Guiding Safe Exploration with Weakest Preconditions
Greg Anderson, Swarat Chaudhuri, Isil Dillig

TL;DR
This paper introduces SPICE, a neurosymbolic method for safe exploration in reinforcement learning that uses symbolic weakest preconditions to ensure safety constraints are met during training, achieving fewer violations and theoretical convergence guarantees.
Contribution
The paper presents SPICE, a novel neurosymbolic approach employing symbolic weakest preconditions for safer exploration in reinforcement learning, with theoretical convergence guarantees.
Findings
Achieves fewer safety violations compared to existing methods.
Maintains comparable performance to current safe learning techniques.
Converges to the optimal safe policy under certain assumptions.
Abstract
In reinforcement learning for safety-critical settings, it is often desirable for the agent to obey safety constraints at all points in time, including during training. We present a novel neurosymbolic approach called SPICE to solve this safe exploration problem. SPICE uses an online shielding layer based on symbolic weakest preconditions to achieve a more precise safety analysis than existing tools without unduly impacting the training process. We evaluate the approach on a suite of continuous control benchmarks and show that it can achieve comparable performance to existing safe learning techniques while incurring fewer safety violations. Additionally, we present theoretical results showing that SPICE converges to the optimal safe policy under reasonable assumptions.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Fault Detection and Control Systems · Reinforcement Learning in Robotics
