Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checking

TL;DR

This paper introduces a metric for quantifying the impact of adversarial attacks on deep reinforcement learning policies' temporal logic properties and presents a model checking approach to verify policy robustness.

Contribution

It proposes a novel metric for adversarial impact on temporal logic properties and a model checking method to verify RL policy robustness.

Findings

The metric effectively guides the creation of optimal adversarial attacks.

The model checking approach accurately assesses policy robustness.

Empirical results demonstrate the metric's effectiveness and the verification method's reliability.

Abstract

Deep Reinforcement Learning (RL) agents are susceptible to adversarial noise in their observations that can mislead their policies and decrease their performance. However, an adversary may be interested not only in decreasing the reward, but also in modifying specific temporal logic properties of the policy. This paper presents a metric that measures the exact impact of adversarial attacks against such properties. We use this metric to craft optimal adversarial attacks. Furthermore, we introduce a model checking method that allows us to verify the robustness of RL policies against adversarial attacks. Our empirical analysis confirms (1) the quality of our metric to craft adversarial attacks against temporal logic properties, and (2) that we are able to concisely assess a system's robustness against attacks.