Whatever Does Not Kill Deep Reinforcement Learning, Makes It Stronger

TL;DR

This paper explores the robustness of deep reinforcement learning agents against adversarial attacks, showing that they can recover and adapt, and that training under attack improves test-time resilience.

Contribution

It demonstrates that deep RL agents can recover from training-time attacks and that adversarial training enhances robustness, comparing exploration strategies for improved resilience.

Findings

DQN agents can recover from noncontiguous training-time attacks.

Policies trained under adversarial conditions are more robust to test-time attacks.

Adversarial training improves overall robustness of deep RL agents.

Abstract

Recent developments have established the vulnerability of deep Reinforcement Learning (RL) to policy manipulation attacks via adversarial perturbations. In this paper, we investigate the robustness and resilience of deep RL to training-time and test-time attacks. Through experimental results, we demonstrate that under noncontiguous training-time attacks, Deep Q-Network (DQN) agents can recover and adapt to the adversarial conditions by reactively adjusting the policy. Our results also show that policies learned under adversarial perturbations are more robust to test-time attacks. Furthermore, we compare the performance of $\epsilon$-greedy and parameter-space noise exploration methods in terms of robustness and resilience against adversarial perturbations.