On the number of bins in a rank histogram

TL;DR

This paper investigates how the choice of bin number affects the interpretation of rank histograms in meteorology, proposing a method to select an optimal bin count that balances visual and statistical assessments.

Contribution

It introduces a new method for choosing the number of bins in rank histograms, improving the detection of calibration issues in forecast systems.

Findings

Choosing fewer bins than ensemble size plus one can improve calibration detection.

The proposed method aligns visual interpretation with formal statistical testing.

Optimal bin number depends on the amount of verification data available.

Abstract

Rank histograms are popular tools for assessing the reliability of meteorological ensemble forecast systems. A reliable forecast system leads to a uniform rank histogram, and deviations from uniformity can indicate miscalibrations. However, the ability to identify such deviations by visual inspection of rank histogram plots crucially depends on the number of bins chosen for the histogram. If too few bins are chosen, the rank histogram is likely to miss miscalibrations; if too many are chosen, even perfectly calibrated forecast systems can yield rank histograms that do not appear uniform. In this paper we address this trade-off and propose a method for choosing the number of bins for a rank histogram. The goal of our method is to select a number of bins such that the intuitive decision whether a histogram is uniform or not is as close as possible to a formal statistical test. Our results indicate that it is often appropriate to choose fewer bins than the usual choice of ensemble size plus one, especially when the number of observations available for verification is small.