Chaos-Free Networks are Stable Recurrent Neural Networks

TL;DR

This paper introduces the Decoupled-Gate Network (DGN), a new stable RNN architecture that guarantees incremental stability, addressing chaos in traditional gated RNNs for nonlinear system identification.

Contribution

It proposes the DGN, a structural variant of the Chaos-Free Network, that unconditionally satisfies delta-ISS, ensuring stability without complex training modifications.

Findings

DGN unconditionally satisfies delta-ISS stability.

Numerical results show DGN maintains modeling capabilities.

CFN satisfies ISS but needs parametric constraints.

Abstract

Gated Recurrent Neural Networks (RNNs) are widely used for nonlinear system identification due to their high accuracy, although they often exhibit complex, chaotic dynamics that are difficult to analyze. This paper investigates the system-theoretic properties of the Chaos-Free Network (CFN), an architecture originally proposed to eliminate the chaotic behavior found in standard gated RNNs. First, we formally prove that the CFN satisfies Input-to-State Stability (ISS) by design. However, we demonstrate that ensuring Incremental ISS (delta-ISS) still requires specific parametric constraints on the CFN architecture. Then, to address this, we introduce the Decoupled-Gate Network (DGN), a novel structural variant of the CFN that removes internal state connections in the gating mechanisms. Finally, we prove that the DGN unconditionally satisfies the delta-ISS property, providing an incrementally stable architecture for identifying nonlinear dynamical systems without requiring complex network training modifications. Numerical results confirm that the DGN maintains the modeling capabilities of standard architectures while adhering to these rigorous stability guarantees.