Fast derivatives of likelihood functionals for ODE based models using adjoint-state method

TL;DR

This paper introduces an efficient adjoint-state method for computing derivatives of likelihood functionals in ODE models, enabling faster and more stable parameter estimation in high-dimensional settings.

Contribution

It derives and analyzes the adjoint-state method for likelihood derivatives in ODE models, with optimized implementation and benchmarking for statistical applications.

Findings

Gradient computation cost is independent of parameter number.

Hessian computation cost scales linearly with parameters.

Method significantly accelerates parameter estimation in ODE models.

Abstract

We consider time series data modeled by ordinary differential equations (ODEs), widespread models in physics, chemistry, biology and science in general. The sensitivity analysis of such dynamical systems usually requires calculation of various derivatives with respect to the model parameters. We employ the adjoint state method (ASM) for efficient computation of the first and the second derivatives of likelihood functionals constrained by ODEs with respect to the parameters of the underlying ODE model. Essentially, the gradient can be computed with a cost (measured by model evaluations) that is independent of the number of the ODE model parameters and the Hessian with a linear cost in the number of the parameters instead of the quadratic one. The sensitivity analysis becomes feasible even if the parametric space is high-dimensional. The main contributions are derivation and rigorous analysis of the ASM in the statistical context, when the discrete data are coupled with the continuous ODE model. Further, we present a highly optimized implementation of the results and its benchmarks on a number of problems. The results are directly applicable in (e.g.) maximum-likelihood estimation or Bayesian sampling of ODE based statistical models, allowing for faster, more stable estimation of parameters of the underlying ODE model.