Model-Embedded Gaussian Process Regression for Parameter Estimation in Dynamical System

TL;DR

This paper introduces a novel Bayesian framework that embeds Gaussian process regression within a one-step approach for efficient parameter estimation in dynamical systems, effectively incorporating physical constraints and derivatives.

Contribution

It develops a model-embedded GPR method that jointly infers parameters and hyperparameters, ensuring physical consistency and providing theoretical convergence guarantees.

Findings

The method accurately estimates parameters in simulated and real data.

It produces low-bias estimates with reliable confidence intervals.

The approach effectively incorporates differential equation constraints.

Abstract

Identifying dynamical system (DS) is a vital task in science and engineering. Traditional methods require numerous calls to the DS solver, rendering likelihood-based or least-squares inference frameworks impractical. For efficient parameter inference, two state-of-the-art techniques are the kernel method for modeling and the "one-step framework" for jointly inferring unknown parameters and hyperparameters. The kernel method is a quick and straightforward technique, but it cannot estimate solutions and their derivatives, which must strictly adhere to physical laws. We propose a model-embedded "one-step" Bayesian framework for joint inference of unknown parameters and hyperparameters by maximizing the marginal likelihood. This approach models the solution and its derivatives using Gaussian process regression (GPR), taking into account smoothness and continuity properties, and treats differential equations as constraints that can be naturally integrated into the Bayesian framework in the linear case. Additionally, we prove the convergence of the model-embedded Gaussian process regression (ME-GPR) for theoretical development. Motivated by Taylor expansion, we introduce a piecewise first-order linearization strategy to handle nonlinear dynamic systems. We derive estimates and confidence intervals, demonstrating that they exhibit low bias and good coverage properties for both simulated models and real data.