Hypothesis Testing for Detecting Outlier Evaluators

TL;DR

This paper introduces a two-stage statistical method to identify outlier evaluators in epidemiological studies, improving the detection of inconsistent measurements among professionals.

Contribution

It proposes a novel two-stage procedure combining regression modeling and hypothesis testing to effectively detect outlier evaluators in epidemiological data.

Findings

High true positive and negative rates in simulation studies

Effective detection of outlier audiologists in real data

Method improves evaluator consistency assessment

Abstract

In epidemiological studies, very often, evaluators obtain measurements of disease outcomes for study participants. In this paper, we propose a two-stage procedure for detecting outlier evaluators. In the first stage, a regression model is fitted to obtain the evaluators' effects. The outlier evaluators are considered as those with different effects compared with the normal evaluators. In the second stage, stepwise hypothesis testings are performed to detect outlier evaluators. The true positive rate and true negative rate of the proposed procedure are assessed in a simulation study. We apply the proposed method to detect potential outlier audiologists among the audiologists who measured hearing threshold levels of the participants in the Audiology Assessment Arm of the Conservation of Hearing Study, which is an epidemiological study for examining risk factors of hearing loss.