Gray-box Adversarial Testing for Control Systems with Machine Learning Component

TL;DR

This paper introduces a gradient-based method combined with randomized search to identify adversarial inputs in neural network-controlled systems, improving testing efficiency for safety-critical applications.

Contribution

It presents a novel gradient-based adversarial testing approach for control systems with neural network components, enhancing verification capabilities.

Findings

Outperforms Simulated Annealing in finding adversarial samples

Effective in testing neural network control systems against system-level requirements

Provides a new tool for safety verification of learning-based controllers

Abstract

Neural Networks (NN) have been proposed in the past as an effective means for both modeling and control of systems with very complex dynamics. However, despite the extensive research, NN-based controllers have not been adopted by the industry for safety critical systems. The primary reason is that systems with learning based controllers are notoriously hard to test and verify. Even harder is the analysis of such systems against system-level specifications. In this paper, we provide a gradient based method for searching the input space of a closed-loop control system in order to find adversarial samples against some system-level requirements. Our experimental results show that combined with randomized search, our method outperforms Simulated Annealing optimization.