Reachable Set Estimation and Verification for Neural Network Models of Nonlinear Dynamic Systems

TL;DR

This paper presents a novel layer-by-layer algorithm for estimating the reachable sets of neural network models of nonlinear systems, enabling formal safety verification for such models in safety-critical applications.

Contribution

It introduces a partition-based, iterative reachable set estimation method for MLP neural networks modeling NARMA systems, facilitating safety verification.

Findings

Over-approximation of reachable sets for MLPs is achievable.

The method effectively verifies safety by checking intersections with unsafe regions.

Numerical examples demonstrate the approach's practicality and accuracy.

Abstract

Neural networks have been widely used to solve complex real-world problems. Due to the complicate, nonlinear, non-convex nature of neural networks, formal safety guarantees for the behaviors of neural network systems will be crucial for their applications in safety-critical systems. In this paper, the reachable set estimation and verification problems for Nonlinear Autoregressive-Moving Average (NARMA) models in the forms of neural networks are addressed. The neural network involved in the model is a class of feed-forward neural networks called Multi-Layer Perceptron (MLP). By partitioning the input set of an MLP into a finite number of cells, a layer-by-layer computation algorithm is developed for reachable set estimation for each individual cell. The union of estimated reachable sets of all cells forms an over-approximation of reachable set of the MLP. Furthermore, an iterative reachable set estimation algorithm based on reachable set estimation for MLPs is developed for NARMA models. The safety verification can be performed by checking the existence of intersections of unsafe regions and estimated reachable set. Several numerical examples are provided to illustrate our approach.