Discrete-Time Stability Analysis of ReLU Feedback Systems via Integral Quadratic Constraints

TL;DR

This paper develops new dynamic integral quadratic constraints (IQCs) for ReLU nonlinearities in discrete-time feedback systems, enabling less conservative stability analysis for neural network-inspired systems.

Contribution

The paper introduces a novel set of hard IQCs for ReLU nonlinearities that extend existing static constraints, improving stability certification accuracy.

Findings

Dynamic IQCs outperform static QCs in stability margins

Numerical results show less conservative stability bounds

IQCs are applicable to neural network feedback systems

Abstract

This paper analyzes internal stability of a discrete-time feedback system with a ReLU nonlinearity. This feedback system is motivated by recurrent neural networks. We first review existing static quadratic constraints (QCs) for slope-restricted nonlinearities. Next, we derive hard integral quadratic constraints (IQCs) for scalar ReLU by using finite impulse filters and structured matrices. These IQCs are combined with a dissipation inequality leading to an LMI condition that certifies internal stability. We show that our new dynamic IQCs for ReLU are a superset of the well-known Zames-Falb IQCs specified for slope-restricted nonlinearities. Numerical results show that the proposed hard IQCs give less conservative stability margins than Zames-Falb multipliers and prior static QC methods, sometimes dramatically so.