Systems of ODEs Parameters Estimation by Using Stochastic Newton-Raphson and Gradient Descent Methods

TL;DR

This paper introduces novel stochastic Newton-Raphson and Gradient Descent methods for efficient parameter estimation in systems of ODEs, outperforming traditional approaches especially in large-scale and complex models.

Contribution

It develops and evaluates stochastic variants of classical optimization methods tailored for ODE parameter estimation, enhancing accuracy and computational efficiency.

Findings

NR converges rapidly to optimal solutions

GD is robust for chaotic systems but may be suboptimal

Proposed methods outperform NLS in error metrics

Abstract

Ordinary differential equations (ODEs) are widely used to describe the time evolution of natural phenomena across various scientific fields. Estimating the parameters of these systems from data is a challenging task, particularly when dealing with nonlinear and high-dimensional models. In this paper, we propose novel methodologies for parameter estimation in systems of ODEs by using the Newton-Raphson (NR) method and Gradient Descent (GD) method. By leveraging the discrete derivative and Taylor expansion, the problem is formulated in a way that enables the application of both methods, allowing for flexible, efficient solutions. Additionally, we extend these approaches to stochastic versions - Stochastic Newton-Raphson (SNR) and Stochastic Gradient Descent (SGD) - to handle large-scale systems with reduced computational cost. The proposed methods are evaluated by using numerical examples, including both linear and nonlinear parameter models, and compare the results to the well-known Nonlinear Least Squares (NLS) method. While NR converges rapidly to the optimal solution, GD demonstrates robustness in handling chaotic systems, though it may occasionally lead to suboptimal results. Overall, the proposed methods provide improved accuracy in parameter estimation for ODE systems, outperforming NLS in terms of error metrics such as bias, mean absolute error (MAE), mean absolute percentage error (MAPE), root mean square error (RMSE), and coefficient of determination R2. These methods offer a valuable tool for fitting ODE models, particularly in scenarios involving big data and complex dynamics.