FMint-SDE: A Multimodal Foundation Model for Accelerating Numerical Simulation of SDEs via Error Correction

TL;DR

FMint-SDE is a multimodal foundation model that uses a transformer architecture to accelerate and improve the accuracy of stochastic differential equation simulations across various scientific domains.

Contribution

Introduces FMint-SDE, a universal error-correction model leveraging multimodal data and in-context learning for large-scale SDE simulation, surpassing traditional methods.

Findings

Achieves better accuracy-efficiency tradeoff than classical solvers.

Demonstrates broad generalization across diverse SDE benchmarks.

Effective across applications in molecular dynamics, finance, and biology.

Abstract

Fast and accurate simulation of dynamical systems is a fundamental challenge across scientific and engineering domains. Traditional numerical integrators often face a trade-off between accuracy and computational efficiency, while existing neural network-based approaches typically require training a separate model for each case. To overcome these limitations, we introduce a novel multi-modal foundation model for large-scale simulations of differential equations: FMint-SDE (Foundation Model based on Initialization for stochastic differential equations). Based on a decoder-only transformer with in-context learning, FMint-SDE leverages numerical and textual modalities to learn a universal error-correction scheme. It is trained using prompted sequences of coarse solutions generated by conventional solvers, enabling broad generalization across diverse systems. We evaluate our models on a suite of challenging SDE benchmarks spanning applications in molecular dynamics, mechanical systems, finance, and biology. Experimental results show that our approach achieves a superior accuracy-efficiency tradeoff compared to classical solvers, underscoring the potential of FMint-SDE as a general-purpose simulation tool for dynamical systems.