A critical look at deep neural network for dynamic system modeling

TL;DR

This paper critically evaluates the effectiveness of deep neural networks, specifically LSTM and CFNN, for dynamic system modeling, revealing their limitations compared to traditional methods like PEM.

Contribution

It provides a comparative analysis highlighting the deficiencies of neural networks in modeling linear systems, challenging their assumed superiority.

Findings

Neural networks fail to produce consistent models even in noise-free conditions.

Neural networks perform worse than PEM in noisy scenarios.

The study questions the reliability of deep neural networks for dynamic system identification.

Abstract

Neural network models become increasingly popular as dynamic modeling tools in the control community. They have many appealing features including nonlinear structures, being able to approximate any functions. While most researchers hold optimistic attitudes towards such models, this paper questions the capability of (deep) neural networks for the modeling of dynamic systems using input-output data. For the identification of linear time-invariant (LTI) dynamic systems, two representative neural network models, Long Short-Term Memory (LSTM) and Cascade Foward Neural Network (CFNN) are compared to the standard Prediction Error Method (PEM) of system identification. In the comparison, four essential aspects of system identification are considered, then several possible defects and neglected issues of neural network based modeling are pointed out. Detailed simulation studies are performed to verify these defects: for the LTI system, both LSTM and CFNN fail to deliver consistent models even in noise-free cases; and they give worse results than PEM in noisy cases.