Proper local scoring rules on discrete sample spaces

TL;DR

This paper characterizes proper local scoring rules on discrete spaces, showing their connection to graph structures and demonstrating their use in methods like pseudo-likelihood and ratio matching.

Contribution

It provides a characterization of proper local scoring rules on finite sample spaces using graph-based homogeneous functions, extending the understanding of local scoring rules.

Findings

Proper local scoring rules depend only on neighborhood probabilities.

Such rules can be characterized via homogeneous functions on graph cliques.

Examples include pseudo-likelihood and ratio matching methods.

Abstract

A scoring rule is a loss function measuring the quality of a quoted probability distribution $Q$ for a random variable $X$, in the light of the realized outcome $x$ of $X$; it is proper if the expected score, under any distribution $P$ for $X$, is minimized by quoting $Q=P$. Using the fact that any differentiable proper scoring rule on a finite sample space ${\mathcal{X}}$ is the gradient of a concave homogeneous function, we consider when such a rule can be local in the sense of depending only on the probabilities quoted for points in a nominated neighborhood of $x$. Under mild conditions, we characterize such a proper local scoring rule in terms of a collection of homogeneous functions on the cliques of an undirected graph on the space ${\mathcal{X}}$. A useful property of such rules is that the quoted distribution $Q$ need only be known up to a scale factor. Examples of the use of such scoring rules include Besag's pseudo-likelihood and Hyv\"{a}rinen's method of ratio matching.