Singhing with Confidence: Visualising the Performance of Confidence Structures

TL;DR

This paper introduces Singh plots, a visual tool for assessing the frequentist coverage of confidence structures, extending traditional confidence intervals to imprecise structures for better inference and prediction.

Contribution

It extends the concept of Singh plots to imprecise confidence structures, providing a visual method to evaluate their coverage properties.

Findings

Singh plots effectively visualize confidence structure coverage.

Application to ProUCL Chebyshev estimator demonstrates utility.

Deviations in Singh plots reveal characteristics of confidence structures.

Abstract

Confidence intervals are an established means of portraying uncertainty about an inferred parameter and can be generated through the use of confidence distributions. For a confidence distribution to be ideal, it must maintain frequentist coverage of the true parameter. This can be represented for a precise distribution by adherence to a cumulative unit uniform distribution, referred to here as a Singh plot. This manuscript extends this to imprecise confidence structures with bounds around the uniform distribution, and describes how deviations convey information regarding the characteristics of confidence structures designed for inference and prediction. This quick visual representation, in a manner similar to ROC curves, aids the development of robust structures and methods that make use of confidence. A demonstration of the utility of Singh plots is provided with an assessment of the coverage of the ProUCL Chebyshev upper confidence limit estimator for the mean of an unknown distribution.