Existence and Uniqueness of Proper Scoring Rules

TL;DR

This paper investigates the mathematical conditions under which proper scoring rules exist and are unique, extending entropy functions in function spaces and analyzing their derivatives and subgradients.

Contribution

It provides a rigorous analytical framework demonstrating the existence and uniqueness of proper scoring rules by extending entropy functions in various function spaces.

Findings

Entropy functions can be extended to open cones with continuous subgradients.

Proper scoring rules are uniquely associated with entropy functions under the framework.

The framework applies to Shannon, Hyv"arinen, and quadratic entropies.

Abstract

To discuss the existence and uniqueness of proper scoring rules one needs to extend the associated entropy functions as sublinear functions to the conic hull of the prediction set. In some natural function spaces, such as the Lebesgue $L^p$-spaces over $\mathbb R^d$, the positive cones have empty interior. Entropy functions defined on such cones have only directional derivatives. Certain entropies may be further extended continuously to open cones in normed spaces containing signed densities. The extended densities are G\^ateaux differentiable except on a negligible set and have everywhere continuous subgradients due to the supporting hyperplane theorem. We introduce the necessary framework from analysis and algebra that allows us to give an affirmative answer to the titular question of the paper. As a result of this, we give a formal sense in which entropy functions have uniquely associated proper scoring rules. We illustrate our framework by studying the derivatives and subgradients of the following three prototypical entropies: Shannon entropy, Hyv\"arinen entropy, and quadratic entropy.