An optimized recursive learning algorithm for three-layer feedforward neural networks for mimo nonlinear system identifications

TL;DR

This paper introduces an optimized recursive learning algorithm for three-layer feedforward neural networks that eliminates the need for selecting a fixed learning rate, improving online system identification.

Contribution

It presents a novel recursive algorithm based on matrix operations and optimization, with proven weak convergence, for improved neural network training in nonlinear system identification.

Findings

Effective in identifying nonlinear dynamic systems

Avoids the challenge of setting a fixed learning rate

Demonstrated through simulation experiments

Abstract

Back-propagation with gradient method is the most popular learning algorithm for feed-forward neural networks. However, it is critical to determine a proper fixed learning rate for the algorithm. In this paper, an optimized recursive algorithm is presented for online learning based on matrix operation and optimization methods analytically, which can avoid the trouble to select a proper learning rate for the gradient method. The proof of weak convergence of the proposed algorithm also is given. Although this approach is proposed for three-layer, feed-forward neural networks, it could be extended to multiple layer feed-forward neural networks. The effectiveness of the proposed algorithms applied to the identification of behavior of a two-input and two-output non-linear dynamic system is demonstrated by simulation experiments.