A scalable, gradient-stable approach to multi-step, nonlinear system identification using first-order methods

TL;DR

This paper introduces a scalable, gradient-stable method for multi-step nonlinear system identification that leverages first-order optimization and automatic differentiation, addressing computational complexity and stability issues.

Contribution

It proposes a novel approach inspired by neural network training, using LPV sensitivity equations and stability analysis to improve nonlinear system identification.

Findings

Method has linear complexity in horizon length and parameter size

Addresses and mitigates exploding gradient issues

Simulation shows effectiveness and efficiency of the approach

Abstract

This paper presents three main contributions to the field of multi-step system identification. First, drawing inspiration from Neural Network (NN) training, it introduces a tool for solving identification problems by leveraging first-order optimization and Automatic Differentiation (AD). The proposed method exploits gradients with respect to the parameters to be identified and leverages Linear Parameter-Varying (LPV) sensitivity equations to model gradient evolution. Second, it demonstrates that the computational complexity of the proposed method is linear in both the multi-step horizon length and the parameter size, ensuring scalability for large identification problems. Third, it formally addresses the "exploding gradient" issue: via a stability analysis of the LPV equations, it derives conditions for a reliable and efficient optimization and identification process for dynamical systems. Simulation results indicate that the proposed method is both effective and efficient, making it a promising tool for future research and applications in nonlinear system identification and non-convex optimization.