A novel Deep Neural Network architecture for non-linear system identification

TL;DR

This paper introduces a new deep neural network architecture designed for non-linear system identification, emphasizing generalization, automatic complexity selection, and suitability for large datasets.

Contribution

The paper proposes a novel DNN architecture with inductive bias and regularization, reducing hyper-parameter tuning and improving scalability for non-linear system identification.

Findings

Effective on large-scale datasets

Reduces hyper-parameter tuning

Enhances generalization in system identification

Abstract

We present a novel Deep Neural Network (DNN) architecture for non-linear system identification. We foster generalization by constraining DNN representational power. To do so, inspired by fading memory systems, we introduce inductive bias (on the architecture) and regularization (on the loss function). This architecture allows for automatic complexity selection based solely on available data, in this way the number of hyper-parameters that must be chosen by the user is reduced. Exploiting the highly parallelizable DNN framework (based on Stochastic optimization methods) we successfully apply our method to large scale datasets.