Classification Loss Function for Parameter Ensembles in Bayesian Hierarchical Models

TL;DR

This paper introduces classification loss functions for parameter ensembles in Bayesian hierarchical models, demonstrating how they can optimize decision-making based on threshold-based classification and relating them to existing loss functions.

Contribution

It formulates weighted and unweighted threshold classification losses and proves their optimal estimators are unit-specific posterior quantiles and medians, respectively.

Findings

Weighted TCL emphasizes false positives or negatives.

Unweighted TCL is minimized by posterior medians.

Relationships between TCL and absolute value loss are established.

Abstract

Parameter ensembles or sets of point estimates constitute one of the cornerstones of modern statistical practice. This is especially the case in Bayesian hierarchical models, where different decision-theoretic frameworks can be deployed to summarize such parameter ensembles. The estimation of these parameter ensembles may thus substantially vary depending on which inferential goals are prioritised by the modeller. In this note, we consider the problem of classifying the elements of a parameter ensemble above or below a given threshold. Two threshold classification losses (TCLs) --weighted and unweighted-- are formulated. The weighted TCL can be used to emphasize the estimation of false positives over false negatives or the converse. We prove that the weighted and unweighted TCLs are optimized by the ensembles of unit-specific posterior quantiles and posterior medians, respectively. In addition, we relate these classification loss functions on parameter ensembles to the concepts of posterior sensitivity and specificity. Finally, we find some relationships between the unweighted TCL and the absolute value loss, which explain why both functions are minimized by posterior medians.