Adversarial Training is Not Ready for Robot Learning

TL;DR

This paper demonstrates that adversarial training, while improving robustness in deep models, introduces significant errors and is not suitable for robot learning applications due to safety concerns.

Contribution

The paper provides a theoretical and experimental analysis showing adversarial training causes specific errors in robot learning, highlighting its current limitations.

Findings

Adversarial training induces transient, systematic, and conditional errors in neural controllers.

Theoretical generalization of adversarial training to safety-domain optimization.

Experimental safety analysis confirms the presence of errors in robot-learning tasks.

Abstract

Adversarial training is an effective method to train deep learning models that are resilient to norm-bounded perturbations, with the cost of nominal performance drop. While adversarial training appears to enhance the robustness and safety of a deep model deployed in open-world decision-critical applications, counterintuitively, it induces undesired behaviors in robot learning settings. In this paper, we show theoretically and experimentally that neural controllers obtained via adversarial training are subjected to three types of defects, namely transient, systematic, and conditional errors. We first generalize adversarial training to a safety-domain optimization scheme allowing for more generic specifications. We then prove that such a learning process tends to cause certain error profiles. We support our theoretical results by a thorough experimental safety analysis in a robot-learning task. Our results suggest that adversarial training is not yet ready for robot learning.