True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics

TL;DR

DynaMix is a novel zero-shot dynamical systems reconstruction model that generalizes to new systems without re-training, outperforming existing models in long-term statistical forecasting and inference speed.

Contribution

Introduces DynaMix, the first zero-shot DSR model capable of out-of-domain generalization using a pre-trained mixture-of-experts architecture.

Findings

DynaMix accurately forecasts long-term evolution of unseen dynamical systems.

DynaMix outperforms existing time series models in long-term statistics.

DynaMix operates with significantly fewer parameters and faster inference times.

Abstract

Complex, temporally evolving phenomena, from climate to brain activity, are governed by dynamical systems (DS). DS reconstruction (DSR) seeks to infer generative surrogate models of these from observed data, reproducing their long-term behavior. Existing DSR approaches require purpose-training for any new system observed, lacking the zero-shot and in-context inference capabilities known from LLMs. Here we introduce DynaMix, a novel multivariate ALRNN-based mixture-of-experts architecture pre-trained for DSR, the first DSR model able to generalize zero-shot to out-of-domain DS. Just from a provided context signal, without any re-training, DynaMix faithfully forecasts the long-term evolution of novel DS where existing time series (TS) foundation models, like Chronos, fail -- at a fraction of the number of parameters (0.1%) and orders of magnitude faster inference times. DynaMix outperforms TS foundation models in terms of long-term statistics, and often also short-term forecasts, even on real-world time series, like traffic or weather data, typically used for training and evaluating TS models, but not at all part of DynaMix' training corpus. We illustrate some of the failure modes of TS models for DSR problems, and conclude that models built on DS principles may bear a huge potential also for advancing the TS prediction field.