Space-Filling Regularization for Robust and Interpretable Nonlinear State Space Models

TL;DR

This paper introduces a novel space-filling regularization method for nonlinear state space models that improves data coverage, interpretability, and robustness by controlling state trajectory distributions during training.

Contribution

It proposes two new regularization techniques based on experimental design principles to enhance state space data distribution in local model architectures.

Findings

Improved data coverage in state space models.

Enhanced interpretability and robustness of models.

Validated on a standard system identification benchmark.

Abstract

The state space dynamics representation is the most general approach for nonlinear systems and often chosen for system identification. During training, the state trajectory can deform significantly leading to poor data coverage of the state space. This can cause significant issues for space-oriented training algorithms which e.g. rely on grid structures, tree partitioning, or similar. Besides hindering training, significant state trajectory deformations also deteriorate interpretability and robustness properties. This paper proposes a new type of space-filling regularization that ensures a favorable data distribution in state space via introducing a data-distribution-based penalty. This method is demonstrated in local model network architectures where good interpretability is a major concern. The proposed approach integrates ideas from modeling and design of experiments for state space structures. This is why we present two regularization techniques for the data point distributions of the state trajectories for local affine state space models. Beyond that, we demonstrate the results on a widely known system identification benchmark.