Towards Robust Neural Networks via Close-loop Control

TL;DR

This paper introduces a novel close-loop control approach to enhance neural network robustness against data perturbations by adaptively generating control signals, leveraging geometric data information, and maintaining performance on clean data.

Contribution

It proposes the first close-loop control method for neural network robustness, connecting optimal control with data geometry to improve resilience against perturbations.

Findings

Improves robustness of neural networks against various perturbations.

Maintains high performance on clean data.

Enhances robustness of trained neural networks.

Abstract

Despite their success in massive engineering applications, deep neural networks are vulnerable to various perturbations due to their black-box nature. Recent study has shown that a deep neural network can misclassify the data even if the input data is perturbed by an imperceptible amount. In this paper, we address the robustness issue of neural networks by a novel close-loop control method from the perspective of dynamic systems. Instead of modifying the parameters in a fixed neural network architecture, a close-loop control process is added to generate control signals adaptively for the perturbed or corrupted data. We connect the robustness of neural networks with optimal control using the geometrical information of underlying data to design the control objective. The detailed analysis shows how the embedding manifolds of state trajectory affect error estimation of the proposed method. Our approach can simultaneously maintain the performance on clean data and improve the robustness against many types of data perturbations. It can also further improve the performance of robustly trained neural networks against different perturbations. To the best of our knowledge, this is the first work that improves the robustness of neural networks with close-loop control.