A Deterministic Approximation to Neural SDEs

TL;DR

This paper introduces a computationally efficient deterministic approximation method for Neural SDEs that enhances uncertainty calibration and prediction accuracy, addressing the high computational cost of traditional Monte Carlo approaches.

Contribution

The paper presents a novel bidimensional moment matching algorithm that approximates Neural SDE transition kernels deterministically, improving efficiency and stability.

Findings

Deterministic approximation matches Monte Carlo calibration quality

Method improves prediction accuracy due to numerical stability

Reduces computational cost compared to sampling-based methods

Abstract

Neural Stochastic Differential Equations (NSDEs) model the drift and diffusion functions of a stochastic process as neural networks. While NSDEs are known to make accurate predictions, their uncertainty quantification properties have been remained unexplored so far. We report the empirical finding that obtaining well-calibrated uncertainty estimations from NSDEs is computationally prohibitive. As a remedy, we develop a computationally affordable deterministic scheme which accurately approximates the transition kernel, when dynamics is governed by a NSDE. Our method introduces a bidimensional moment matching algorithm: vertical along the neural net layers and horizontal along the time direction, which benefits from an original combination of effective approximations. Our deterministic approximation of the transition kernel is applicable to both training and prediction. We observe in multiple experiments that the uncertainty calibration quality of our method can be matched by Monte Carlo sampling only after introducing high computational cost. Thanks to the numerical stability of deterministic training, our method also improves prediction accuracy.