On the stability properties of Gated Recurrent Units neural networks

TL;DR

This paper establishes conditions to ensure the stability of Gated Recurrent Units (GRUs), useful for verifying or enforcing stability during training, demonstrated on a nonlinear control benchmark.

Contribution

It provides the first set of sufficient nonlinear inequalities on weights to guarantee ISS and { extdelta}ISS for GRUs, applicable to both single and multi-layer architectures.

Findings

Conditions can verify stability of trained GRUs

Stability constraints can be integrated into training

Demonstrated on a nonlinear control benchmark

Abstract

The goal of this paper is to provide sufficient conditions for guaranteeing the Input-to-State Stability (ISS) and the Incremental Input-to-State Stability ({\delta}ISS) of Gated Recurrent Units (GRUs) neural networks. These conditions, devised for both single-layer and multi-layer architectures, consist of nonlinear inequalities on network's weights. They can be employed to check the stability of trained networks, or can be enforced as constraints during the training procedure of a GRU. The resulting training procedure is tested on a Quadruple Tank nonlinear benchmark system, showing satisfactory modeling performances.