Comparison of Uncertainty Quantification with Deep Learning in Time Series Regression

TL;DR

This paper compares various uncertainty quantification methods in deep learning for time series regression, focusing on meteorological data, to evaluate their robustness and how well they meet expected behaviors like increased uncertainty with longer horizons.

Contribution

It provides a comparative analysis of uncertainty estimation techniques in deep learning for time series forecasting, highlighting their strengths and limitations in real-world meteorological applications.

Findings

Different methods vary in how well they meet expected uncertainty behaviors.

Some methods show robustness in forecasting meteorological data.

The evaluation highlights the importance of choosing appropriate uncertainty measures.

Abstract

Increasingly high-stakes decisions are made using neural networks in order to make predictions. Specifically, meteorologists and hedge funds apply these techniques to time series data. When it comes to prediction, there are certain limitations for machine learning models (such as lack of expressiveness, vulnerability of domain shifts and overconfidence) which can be solved using uncertainty estimation. There is a set of expectations regarding how uncertainty should ``behave". For instance, a wider prediction horizon should lead to more uncertainty or the model's confidence should be proportional to its accuracy. In this paper, different uncertainty estimation methods are compared to forecast meteorological time series data and evaluate these expectations. The results show how each uncertainty estimation method performs on the forecasting task, which partially evaluates the robustness of predicted uncertainty.