# Why do some probabilistic forecasts lack reliability?

## TL;DR

This paper proves that maximizing the Peirce skill score at the climatological base rate threshold is necessary for reliable probabilistic binary forecasts, providing guidance for forecast system development.

## Contribution

It establishes a mathematical condition linking forecast reliability to maximizing PSS at the climatological base rate threshold, which is crucial for developing reliable probabilistic forecasts.

## Key findings

- Maximizing PSS at the climatological base rate threshold is necessary for reliability.
- The condition is validated with synthetic data and existing solar flare forecast models.
- Forecast systems not satisfying this condition cannot be reliable.

## Abstract

In this work, we investigate the reliability of the probabilistic binary forecast. We mathematically prove that a necessary, but not sufficient, condition for achieving a reliable probabilistic forecast is maximizing the Peirce skill score (PSS) at the threshold probability of the climatological base rate. The condition is confirmed by using artificially synthesized forecast-outcome pair data and previously published probabilistic solar flare forecast models. The condition gives a partial answer as to why some probabilistic forecast system lack reliability, because the system, which does not satisfy the proved condition, can never be reliable. Therefore, the proved condition is very important for the developers of a probabilistic forecast system. The result implies that those who want to develop a reliable probabilistic forecast system must adjust or train the system so as to maximize PSS near the threshold probability of the climatological base rate.