Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions

TL;DR

This paper introduces a new training method for Neural SDEs called Finite Dimensional Matching (FDM), which is more efficient and effective than previous adversarial or kernel-based approaches, by leveraging proper scoring rules and the Markov property.

Contribution

The paper proposes a novel FDM training approach for Neural SDEs based on proper scoring rules, reducing computational complexity and improving generative performance.

Findings

FDM reduces training complexity from O(D^2) to O(D).

FDM outperforms existing methods in efficiency and quality.

Theoretical identification of proper scoring rules for continuous Markov processes.

Abstract

Neural Stochastic Differential Equations (Neural SDEs) have emerged as powerful mesh-free generative models for continuous stochastic processes, with critical applications in fields such as finance, physics, and biology. Previous state-of-the-art methods have relied on adversarial training, such as GANs, or on minimizing distance measures between processes using signature kernels. However, GANs suffer from issues like instability, mode collapse, and the need for specialized training techniques, while signature kernel-based methods require solving linear PDEs and backpropagating gradients through the solver, whose computational complexity scales quadratically with the discretization steps. In this paper, we identify a novel class of strictly proper scoring rules for comparing continuous Markov processes. This theoretical finding naturally leads to a novel approach called Finite Dimensional Matching (FDM) for training Neural SDEs. Our method leverages the Markov property of SDEs to provide a computationally efficient training objective. This scoring rule allows us to bypass the computational overhead associated with signature kernels and reduces the training complexity from $O(D^2)$ to $O(D)$ per epoch, where $D$ represents the number of discretization steps of the process. We demonstrate that FDM achieves superior performance, consistently outperforming existing methods in terms of both computational efficiency and generative quality.