ODEFormer: Symbolic Regression of Dynamical Systems with Transformers

TL;DR

ODEFormer is a transformer-based model capable of inferring symbolic multidimensional ODE systems from single solution trajectories, outperforming existing methods in accuracy, robustness, and speed.

Contribution

This work introduces ODEFormer, the first transformer model for symbolic regression of multidimensional ODEs from limited observational data.

Findings

Outperforms existing methods in accuracy and robustness

Handles noisy and irregularly sampled data effectively

Provides faster inference than previous approaches

Abstract

We introduce ODEFormer, the first transformer able to infer multidimensional ordinary differential equation (ODE) systems in symbolic form from the observation of a single solution trajectory. We perform extensive evaluations on two datasets: (i) the existing "Strogatz" dataset featuring two-dimensional systems; (ii) ODEBench, a collection of one- to four-dimensional systems that we carefully curated from the literature to provide a more holistic benchmark. ODEFormer consistently outperforms existing methods while displaying substantially improved robustness to noisy and irregularly sampled observations, as well as faster inference. We release our code, model and benchmark dataset publicly.