Foundation Inference Models for Ordinary Differential Equations

TL;DR

FIM-ODE is a pretrained model that efficiently infers ODE vector fields from noisy data in a single pass, outperforming existing methods and enabling easy adaptation without extensive ML expertise.

Contribution

We introduce FIM-ODE, a novel pretrained foundation model for ODE inference that achieves zero-shot performance and fast fine-tuning, simplifying the process compared to prior approaches.

Findings

FIM-ODE matches or exceeds the performance of recent symbolic baselines.

Pretraining enables fast, stable adaptation to new ODEs.

FIM-ODE outperforms modern neural and Gaussian process baselines.

Abstract

Ordinary differential equations (ODEs) are central to scientific modelling, but inferring their vector fields from noisy trajectories remains challenging. Current approaches such as symbolic regression, Gaussian process (GP) regression, and Neural ODEs often require complex training pipelines and substantial machine learning expertise, or they depend strongly on system-specific prior knowledge. We propose FIM-ODE, a pretrained Foundation Inference Model that amortises low-dimensional ODE inference by predicting the vector field directly from noisy trajectory data in a single forward pass. We pretrain FIM-ODE on a prior distribution over ODEs with low-degree polynomial vector fields and represent the target field with neural operators. FIM-ODE achieves strong zero-shot performance, matching and often improving upon ODEFormer, a recent pretrained symbolic baseline, across a range of regimes despite using a simpler pretraining prior distribution. Pretraining also provides a strong initialisation for finetuning, enabling fast and stable adaptation that outperforms modern neural and GP baselines without requiring machine learning expertise.