Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey

TL;DR

This survey reviews adversarial attack and training methods to enhance the robustness and trustworthiness of deep reinforcement learning agents in complex environments.

Contribution

It provides a systematic categorization and comparison of current adversarial attack and training techniques in DRL.

Findings

Adversarial training improves DRL robustness against environmental perturbations

Different attack and training methods have varying effectiveness and operational mechanisms

The survey highlights gaps and future directions in robust DRL research

Abstract

Deep Reinforcement Learning (DRL) is a subfield of machine learning for training autonomous agents that take sequential actions across complex environments. Despite its significant performance in well-known environments, it remains susceptible to minor condition variations, raising concerns about its reliability in real-world applications. To improve usability, DRL must demonstrate trustworthiness and robustness. A way to improve the robustness of DRL to unknown changes in the environmental conditions and possible perturbations is through Adversarial Training, by training the agent against well-suited adversarial attacks on the observations and the dynamics of the environment. Addressing this critical issue, our work presents an in-depth analysis of contemporary adversarial attack and training methodologies, systematically categorizing them and comparing their objectives and operational mechanisms.