A probabilistic model for the numerical solution of initial value problems

TL;DR

This paper introduces a probabilistic framework for solving initial value problems in ODEs by interpreting classical numerical methods as inference on latent Gaussian processes, enabling uncertainty quantification.

Contribution

It formulates IVP solvers as inference problems on Gaussian processes, connecting classical methods to probabilistic models and providing a basis for uncertainty estimation.

Findings

Probabilistic formulation links classical ODE solvers to Gaussian process inference.

Certain methods correspond exactly to generalized linear, Runge-Kutta, and Nordsieck methods.

Framework allows incorporation of prior information and uncertainty quantification.

Abstract

Like many numerical methods, solvers for initial value problems (IVPs) on ordinary differential equations estimate an analytically intractable quantity, using the results of tractable computations as inputs. This structure is closely connected to the notion of inference on latent variables in statistics. We describe a class of algorithms that formulate the solution to an IVP as inference on a latent path that is a draw from a Gaussian process probability measure (or equivalently, the solution of a linear stochastic differential equation). We then show that certain members of this class are connected precisely to generalized linear methods for ODEs, a number of Runge--Kutta methods, and Nordsieck methods. This probabilistic formulation of classic methods is valuable in two ways: analytically, it highlights implicit prior assumptions favoring certain approximate solutions to the IVP over others, and gives a precise meaning to the old observation that these methods act like filters. Practically, it endows the classic solvers with `docking points' for notions of uncertainty and prior information about the initial value, the value of the ODE itself, and the solution of the problem.