Learning solutions of parameterized stiff ODEs using Gaussian processes

TL;DR

This paper introduces a reparameterization method for solutions of parameterized stiff ODEs to improve Gaussian process surrogate modeling by making solutions more stationary, thereby enhancing approximation accuracy with minimal computational cost.

Contribution

The authors propose a simple preprocessing reparameterization technique that enhances GP performance on stiff ODE solutions without altering existing GP implementations.

Findings

Reparameterization improves GP approximation accuracy for stiff ODE solutions.

The method requires minimal additional computational overhead.

Examples demonstrate significant benefits in surrogate modeling of parameterized stiff ODEs.

Abstract

Stiff ordinary differential equations (ODEs) play an important role in many scientific and engineering applications. Often, the dependence of the solution of the ODE on additional parameters is of interest, e.g.\ when dealing with uncertainty quantification or design optimization. Directly studying this dependence can quickly become too computationally expensive, such that cheaper surrogate models approximating the solution are of interest. One popular class of surrogate models are Gaussian processes (GPs). They perform well when approximating stationary functions, functions which have a similar level of variation along any given parameter direction, however solutions to stiff ODEs are often characterized by a mixture of regions of rapid and slow variation along the time axis and when dealing with such nonstationary functions, GP performance frequently degrades drastically. We therefore aim to reparameterize stiff ODE solutions based on the available data, to make them appear more stationary and hence recover good GP performance. This approach comes with minimal computational overhead and requires no internal changes to the GP implementation, as it can be seen as a separate preprocessing step. We illustrate the achieved benefits using multiple examples.