Analytical Gradient and Hessian Evaluation for System Identification using State-Parameter Transition Tensors

TL;DR

This paper introduces an analytical method using Einstein notation to compute gradients and Hessians for system identification, improving robustness over finite difference methods and demonstrating effectiveness on real datasets.

Contribution

It presents a novel analytical approach to derive gradients and Hessians for system identification using state-parameter transition tensors, enhancing accuracy and robustness.

Findings

Analytical gradients and Hessians outperform finite difference methods.

The proposed method shows improved robustness in identifying system parameters.

Validated on real dynamic system datasets with superior results.

Abstract

In this work, the Einstein notation is utilized to synthesize state and parameter transition matrices, by solving a set of ordinary differential equations. Additionally, for the system identification problem, it has been demonstrated that the gradient and Hessian of a cost function can be analytically constructed using the same matrix and tensor metrics. A general gradientbased optimization problem is then posed to identify unknown system parameters and unknown initial conditions. Here, the analytical gradient and Hessian of the cost function are derived using these state and parameter transition matrices. The more robust performance of the proposed method for identifying unknown system parameters and unknown initial conditions over an existing conventional quasi-Newton method-based system identification toolbox (available in MATLAB) is demonstrated by using two widely used benchmark datasets from real dynamic systems. In the existing toolbox, gradient and Hessian information, which are derived using a finite difference method, are more susceptible to numerical errors compared to the analytical approach presented. Keywords: Gradient-based Optimization, Transition matrix and tensors, Gradient and Hessian, System identification.