Possible, Yes; Ignorant, Perhaps: A Scorecard for Possibilistic Forecasts

TL;DR

This paper introduces a verification framework for possibilistic forecasts using a five-number scorecard that diagnoses various aspects like accuracy, sharpness, confidence, and dominance, enhancing understanding beyond traditional probabilistic methods.

Contribution

It develops a novel scorecard and verification framework for possibilistic forecasts, integrating possibilistic, probabilistic, and categorical assessments.

Findings

The scorecard effectively diagnoses forecast accuracy and ignorance.

Possibility-to-probability conversion preserves ignorance in scoring.

Diagnostic visualizations improve interpretation of possibilistic forecasts.

Abstract

Probabilistic forecasts must sum to unity and cannot express ``I don't know.'' Possibility theory relaxes this constraint: a subnormal distribution explicitly measures how much of the plausibility budget remains unassigned, ignorance signal that probability cannot represent. This paper develops a verification framework for such forecasts, centred on a five-number scorecard that separately diagnoses whether the forecast pointed at the right outcome (depth-of-truth), how sharply (diffuseness, support margin), how confidently (ignorance), and how dominantly (conditional necessity). A possibility-to-probability conversion preserves ignorance for familiar frequency-based scoring; categorical threshold scores (POD, FAR, CSI, etc.) connect to operational practice. Together, these three complementary facets -- possibilistic, probabilistic, and categorical -- expose failure modes invisible to any single metric. Storm Prediction Center convective outlook categories serve as the running example throughout; a synthetic reforecast demonstrates diagnostic visualisations and scorecard interpretation. Ignorance is better expressed than repressed.