Deep Subspace Encoders for Nonlinear System Identification

TL;DR

This paper introduces a novel neural network-based method for nonlinear system identification that improves stability, data efficiency, and consistency by using a truncated prediction loss and a subspace encoder for state estimation.

Contribution

The paper proposes a new approach combining a truncated prediction loss with a subspace encoder, enhancing stability and efficiency in neural network-based system identification.

Findings

Method is locally consistent under mild conditions.

Increases optimization stability and data efficiency.

Achieves state-of-the-art benchmark results.

Abstract

Using Artificial Neural Networks (ANN) for nonlinear system identification has proven to be a promising approach, but despite of all recent research efforts, many practical and theoretical problems still remain open. Specifically, noise handling and models, issues of consistency and reliable estimation under minimisation of the prediction error are the most severe problems. The latter comes with numerous practical challenges such as explosion of the computational cost in terms of the number of data samples and the occurrence of instabilities during optimization. In this paper, we aim to overcome these issues by proposing a method which uses a truncated prediction loss and a subspace encoder for state estimation. The truncated prediction loss is computed by selecting multiple truncated subsections from the time series and computing the average prediction loss. To obtain a computationally efficient estimation method that minimizes the truncated prediction loss, a subspace encoder represented by an artificial neural network is introduced. This encoder aims to approximate the state reconstructability map of the estimated model to provide an initial state for each truncated subsection given past inputs and outputs. By theoretical analysis, we show that, under mild conditions, the proposed method is locally consistent, increases optimization stability, and achieves increased data efficiency by allowing for overlap between the subsections. Lastly, we provide practical insights and user guidelines employing a numerical example and state-of-the-art benchmark results.