Predicting Ordinary Differential Equations with Transformers

TL;DR

This paper introduces a transformer-based model capable of accurately recovering scalar ordinary differential equations from noisy, irregular data, demonstrating superior or comparable performance to existing methods and efficient scalability after pretraining.

Contribution

The authors present a novel transformer-based sequence-to-sequence approach for symbolic ODE recovery, with demonstrated scalability and high accuracy across diverse scenarios.

Findings

Performs better or on par with existing methods

Efficient inference after pretraining

Scalable to new observed solutions

Abstract

We develop a transformer-based sequence-to-sequence model that recovers scalar ordinary differential equations (ODEs) in symbolic form from irregularly sampled and noisy observations of a single solution trajectory. We demonstrate in extensive empirical evaluations that our model performs better or on par with existing methods in terms of accurate recovery across various settings. Moreover, our method is efficiently scalable: after one-time pretraining on a large set of ODEs, we can infer the governing law of a new observed solution in a few forward passes of the model.