Deep Uncertainty Quantification: A Machine Learning Approach for Weather Forecasting

TL;DR

This paper introduces a deep learning-based method for weather forecasting that combines uncertainty quantification with improved accuracy, outperforming traditional numerical models on a public dataset.

Contribution

It proposes a novel deep uncertainty quantification framework with a new loss function, integrating prior knowledge and ensemble strategies for enhanced weather prediction.

Findings

Significant accuracy improvement over NWP (47.76%)

NLE loss outperforms MSE and MAE in generalization

Achieved 2nd place in a weather forecasting competition

Abstract

Weather forecasting is usually solved through numerical weather prediction (NWP), which can sometimes lead to unsatisfactory performance due to inappropriate setting of the initial states. In this paper, we design a data-driven method augmented by an effective information fusion mechanism to learn from historical data that incorporates prior knowledge from NWP. We cast the weather forecasting problem as an end-to-end deep learning problem and solve it by proposing a novel negative log-likelihood error (NLE) loss function. A notable advantage of our proposed method is that it simultaneously implements single-value forecasting and uncertainty quantification, which we refer to as deep uncertainty quantification (DUQ). Efficient deep ensemble strategies are also explored to further improve performance. This new approach was evaluated on a public dataset collected from weather stations in Beijing, China. Experimental results demonstrate that the proposed NLE loss significantly improves generalization compared to mean squared error (MSE) loss and mean absolute error (MAE) loss. Compared with NWP, this approach significantly improves accuracy by 47.76%, which is a state-of-the-art result on this benchmark dataset. The preliminary version of the proposed method won 2nd place in an online competition for daily weather forecasting.