Do Androids Dream of Electric Fences? Safety-Aware Reinforcement Learning with Latent Shielding

TL;DR

This paper introduces latent shielding, a safety-aware reinforcement learning method that uses internal environment representations to predict and avoid unsafe trajectories in high-dimensional settings.

Contribution

It proposes a novel latent shielding approach that leverages learned internal representations for safety in high-dimensional reinforcement learning environments.

Findings

Improved safety adherence in complex environments

Effective prediction of unsafe trajectories

Outperforms existing safety methods

Abstract

The growing trend of fledgling reinforcement learning systems making their way into real-world applications has been accompanied by growing concerns for their safety and robustness. In recent years, a variety of approaches have been put forward to address the challenges of safety-aware reinforcement learning; however, these methods often either require a handcrafted model of the environment to be provided beforehand, or that the environment is relatively simple and low-dimensional. We present a novel approach to safety-aware deep reinforcement learning in high-dimensional environments called latent shielding. Latent shielding leverages internal representations of the environment learnt by model-based agents to "imagine" future trajectories and avoid those deemed unsafe. We experimentally demonstrate that this approach leads to improved adherence to formally-defined safety specifications.