Foundation Inference Models for Markov Jump Processes

TL;DR

This paper presents a zero-shot neural network approach for inferring Markov jump processes from noisy, sparse data across various scientific applications, without needing dataset-specific training.

Contribution

The authors introduce a novel methodology combining probabilistic simulation and neural networks for zero-shot inference of MJPs in diverse state spaces.

Findings

Model infers MJPs across different dimensions without fine-tuning.

Performs comparably to state-of-the-art models after training.

Effective on biological and physical systems data.

Abstract

Markov jump processes are continuous-time stochastic processes which describe dynamical systems evolving in discrete state spaces. These processes find wide application in the natural sciences and machine learning, but their inference is known to be far from trivial. In this work we introduce a methodology for zero-shot inference of Markov jump processes (MJPs), on bounded state spaces, from noisy and sparse observations, which consists of two components. First, a broad probability distribution over families of MJPs, as well as over possible observation times and noise mechanisms, with which we simulate a synthetic dataset of hidden MJPs and their noisy observation process. Second, a neural network model that processes subsets of the simulated observations, and that is trained to output the initial condition and rate matrix of the target MJP in a supervised way. We empirically demonstrate that one and the same (pretrained) model can infer, in a zero-shot fashion, hidden MJPs evolving in state spaces of different dimensionalities. Specifically, we infer MJPs which describe (i) discrete flashing ratchet systems, which are a type of Brownian motors, and the conformational dynamics in (ii) molecular simulations, (iii) experimental ion channel data and (iv) simple protein folding models. What is more, we show that our model performs on par with state-of-the-art models which are finetuned to the target datasets.