Robust Optimization Framework for Training Shallow Neural Networks Using Reachability Method

TL;DR

This paper introduces a robust training framework for shallow neural networks that leverages reachability analysis and semidefinite programming to enhance robustness against input perturbations, demonstrated on a robot arm example.

Contribution

It develops a novel robust optimization method based on reachability analysis and semidefinite programming for training shallow neural networks.

Findings

Improved robustness against input perturbations.

Trade-off between robustness and training accuracy.

Effective on a robot arm model learning task.

Abstract

In this paper, a robust optimization framework is developed to train shallow neural networks based on reachability analysis of neural networks. To characterize noises of input data, the input training data is disturbed in the description of interval sets. Interval-based reachability analysis is then performed for the hidden layer. With the reachability analysis results, a robust optimization training method is developed in the framework of robust least-square problems. Then, the developed robust least-square problem is relaxed to a semidefinite programming problem. It has been shown that the developed robust learning method can provide better robustness against perturbations at the price of loss of training accuracy to some extent. At last, the proposed method is evaluated on a robot arm model learning example.