Univariate ReLU neural network and its application in nonlinear system identification

TL;DR

This paper introduces a univariate ReLU neural network for modeling nonlinear dynamic systems, demonstrating its effectiveness on a Bouc-Wen hysteretic system with fewer parameters than traditional networks.

Contribution

It proposes a simple, single-hidden-layer UReLU neural network with a novel initialization method for nonlinear system identification, reducing parameters while maintaining accuracy.

Findings

Effective modeling of nonlinear systems with fewer parameters.

Successful application to Bouc-Wen hysteretic system.

Good approximation performance verified through simulations.

Abstract

ReLU (rectified linear units) neural network has received significant attention since its emergence. In this paper, a univariate ReLU (UReLU) neural network is proposed to both modelling the nonlinear dynamic system and revealing insights about the system. Specifically, the neural network consists of neurons with linear and UReLU activation functions, and the UReLU functions are defined as the ReLU functions respect to each dimension. The UReLU neural network is a single hidden layer neural network, and the structure is relatively simple. The initialization of the neural network employs the decoupling method, which provides a good initialization and some insight into the nonlinear system. Compared with normal ReLU neural network, the number of parameters of UReLU network is less, but it still provide a good approximation of the nonlinear dynamic system. The performance of the UReLU neural network is shown through a Hysteretic benchmark system: the Bouc-Wen system. Simulation results verify the effectiveness of the proposed method.