Accurate and Reliable Forecasting using Stochastic Differential Equations

TL;DR

This paper introduces SDE-HNN, a novel heteroscedastic neural network utilizing stochastic differential equations to improve uncertainty estimation and predictive accuracy in regression tasks, with theoretical guarantees and enhanced stability.

Contribution

It develops a new neural SDE framework for heteroscedastic neural networks, providing theoretical analysis, improved numerical methods, and systematic uncertainty evaluation metrics.

Findings

Outperforms state-of-the-art baselines in predictive accuracy

Provides well-calibrated and sharp prediction intervals

Demonstrates theoretical existence and uniqueness of neural SDE solutions

Abstract

It is critical yet challenging for deep learning models to properly characterize uncertainty that is pervasive in real-world environments. Although a lot of efforts have been made, such as heteroscedastic neural networks (HNNs), little work has demonstrated satisfactory practicability due to the different levels of compromise on learning efficiency, quality of uncertainty estimates, and predictive performance. Moreover, existing HNNs typically fail to construct an explicit interaction between the prediction and its associated uncertainty. This paper aims to remedy these issues by developing SDE-HNN, a new heteroscedastic neural network equipped with stochastic differential equations (SDE) to characterize the interaction between the predictive mean and variance of HNNs for accurate and reliable regression. Theoretically, we show the existence and uniqueness of the solution to the devised neural SDE. Moreover, based on the bias-variance trade-off for the optimization in SDE-HNN, we design an enhanced numerical SDE solver to improve the learning stability. Finally, to more systematically evaluate the predictive uncertainty, we present two new diagnostic uncertainty metrics. Experiments on the challenging datasets show that our method significantly outperforms the state-of-the-art baselines in terms of both predictive performance and uncertainty quantification, delivering well-calibrated and sharp prediction intervals.