Generating Probabilities From Numerical Weather Forecasts by Logistic Regression

TL;DR

This paper explores logistic regression models, including regularization techniques like lasso, to convert numerical weather forecasts into reliable probability predictions for binary weather events.

Contribution

It introduces a regularized logistic modeling approach, including a logit lasso, for effective probability forecast generation and automatic model reduction in weather prediction.

Findings

Regularized logistic models improve probability forecasts.

Lasso-based models identify and discard less important inputs.

Efficient model assessment and selection methods are presented.

Abstract

Logistic models are studied as a tool to convert output from numerical weather forecasting systems (deterministic and ensemble) into probability forecasts for binary events. A logistic model obtains by putting the logarithmic odds ratio equal to a linear combination of the inputs. As any statistical model, logistic models will suffer from over-fitting if the number of inputs is comparable to the number of forecast instances. Computational approaches to avoid over-fitting by regularisation are discussed, and efficient approaches for model assessment and selection are presented. A logit version of the so called lasso, which is originally a linear tool, is discussed. In lasso models, less important inputs are identified and discarded, thereby providing an efficient and automatic model reduction procedure. For this reason, lasso models are particularly appealing for diagnostic purposes.