Interpretation and inference for altmetric indicators arising from sparse data statistics

TL;DR

This paper reviews and corrects the statistical interpretation of the MHq altmetric indicator, introduces a new MHRR measure, and demonstrates improved inference accuracy through simulation studies.

Contribution

It corrects the interpretation and variance formula of MHq, and proposes a new indicator MHRR for better altmetric data analysis.

Findings

Corrected variance formula for MHq improves confidence intervals.

Simulation shows superior performance of the new variance estimator.

Introduced MHRR as a new meaningful indicator for altmetrics.

Abstract

In 2018 Bornmann and Haunschild (2018a) introduced a new indicator called the Mantel-Haenszel quotient (MHq) to measure alternative metrics (or altmetrics) of scientometric data. In this article we review the Mantel-Haenszel statistics, point out two errors in the literature, and introduce a new indicator. First, we correct the interpretation of MHq and mention that it is still a meaningful indicator. Second, we correct the variance formula for MHq, which leads to narrower confidence intervals. A simulation study shows the superior performance of our variance estimator and confidence intervals. Since MHq does not match its original description in the literature, we propose a new indicator, the Mantel-Haenszel row risk ratio (MHRR), to meet that need. Interpretation and statistical inference for MHRR are discussed. For both MHRR and MHq, a value greater (less) than one means performance is better (worse) than in the reference set called the world.