The consequences of using statistical tests on proxy measurements in place of gold standard measurements: an application to magnetic resonance spectroscopy

TL;DR

This paper shows how using less accurate proxy measurements instead of gold-standard ones can lead to misleading statistical results in biomedical studies, using MRS as an example.

Contribution

The paper introduces a novel analysis of how proxy measurement correlation affects false positive and false negative rates in statistical testing.

Findings

Moderate correlations between proxy and gold-standard measurements can cause large increases in false positive rates.

Imperfect correlation between proxy and gold-standard measurements reduces statistical power, increasing false negative rates.

Small biases in proxy measurements can significantly distort statistical outcomes.

Abstract

The use of proxy measurements in biomedical science is ubiquitous, due to the infeasibility or unavailability of gold-standard (i.e., most precise, accurate, and/or validated) measurements. For example, in magnetic resonance spectroscopy (MRS), short-echo time (TE) sequences are frequently employed to estimate difficult-to-measure metabolites such as GABA, despite J-difference editing being the recommended gold-standard due to improved metabolic specificity. This work investigates the critical relationship between the correlation of proxy and gold-standard measurements and the associated false positive (FPR) and false negative (FNR) rates of statistical tests performed on proxy measurements. Through statistical simulations, we demonstrate that even moderately high correlations (0.6–0.7), reported in the literature for short-TE vs. J-edited estimated GABA, can lead to drastically inflated FPRs and FNRs. We show that these rates are highly sensitive to the magnitude of differential bias in the proxy measurement (δ) and the underlying true effect size (Δ). For instance, a small, unmeasured bias in short-TE estimated GABA, potentially arising from macromolecule contamination, can substantially inflate FPRs. Conversely, imperfect correlation can substantially reduce statistical power, leading to high FNRs, which may explain some discrepancies within the literature. Although this work focuses specifically on the relationship between short-TE and MEGA-edited GABA, the arguments presented here apply more broadly to other difficult-to-measure metabolites in MRS (e.g., glutathione, 2-hydroxyglutarate), or generally to any circumstance where statistical tests are performed on the readily available proxy measurements in place of gold-standard measurements.

Figures