Simultaneous confidence sets for ranks using the partitioning principle - Technical report

TL;DR

This paper introduces a novel method for constructing simultaneous confidence intervals for ranks of institutions using a partitioning principle and likelihood ratio tests, with improved performance over traditional methods especially when centers are close or data uncertainty is high.

Contribution

The paper presents a new likelihood ratio-based partitioning method for confidence sets of ranks, including algorithms to reduce computational complexity and comparisons with Tukey's HSD test.

Findings

Likelihood ratio test improves rank confidence intervals when centers are close.

Polynomial algorithm provides good bracketing with reduced complexity.

Simulation shows better performance of LR test over Tukey's HSD in certain conditions.

Abstract

Ranking institutions such as medical centers or universities is based on an indicator accompanied with an uncertainty measure such as a standard deviation, and confidence intervals should be calculated to assess the quality of these ranks. We consider the problem of constructing simultaneous confidence intervals for the ranks of centers based on an observed sample. We present in this paper a novel method based on multiple testing which uses the partitioning principle and employs the likelihood ratio (LR) test on the partitions. The complexity of the algorithm is super exponential. We present several ways and shortcuts to reduce this complexity. We provide also a polynomial algorithm which produces a very good bracketing for the multiple testing by linearizing the critical value of the LR test. We show that Tukey's Honest Significant Difference (HSD) test can be written as a partitioning procedure. The new methodology has promising properties in the sens that it opens the door in a simple and easy way to construct new methods which may trade the exponential complexity with power of the test or vice versa. In comparison to Tukey's HSD test, the LR test seems to give better results when the centers are close to each others or the uncertainty in the data is high which is confirmed during a simulation study.