An improvement of Tukey's HSD with application to ranking institutions

TL;DR

This paper introduces a novel, valid method based on Tukey's HSD for constructing simultaneous confidence intervals for ranks of institutions, improving interval length control and familywise error rate, with practical applications and software implementation.

Contribution

It presents the first valid simultaneous confidence intervals for ranks using Tukey's HSD and introduces a new variant with improved interval lengths and error control.

Findings

The new method produces shorter confidence intervals for ranks.

It ensures familywise error control in rank confidence intervals.

Validated through simulations and real hospital data.

Abstract

When a ranking of institutions such as medical centers or universities is based on an indicator provided with a standard error, confidence intervals should be calculated to assess the quality of these ranks. We consider the problem of constructing simultaneous confidence intervals (CIs) for the ranks of centers based on an observed sample. We present a novel method based on Tukey's honest significant difference test (HSD) which is the first method to produce valid simultaneous CIs for ranks. Moreover, we introduce a new variant of Tukey's HSD based on the sequential rejection principle. The new algorithm ensures familywise error control, and produces simultaneous confidence intervals for the ranks uniformly shorter than those provided by Tukey's HSD for the same level of significance. We illustrate the method through both simulations and real data analysis from 64 hospitals in the Netherlands. Software for our new methods is available online in package \texttt{ICRanks} downloadable from CRAN. Supplementary materials include supplementary R code for the simulations and proofs of the propositions presented in this paper.