A no-gold-standard technique to objectively evaluate quantitative imaging methods using patient data: Theory

TL;DR

This paper introduces a new statistical method to objectively evaluate quantitative imaging techniques directly from patient data without needing a gold standard, by modeling correlated measurement noise.

Contribution

It develops a maximum-likelihood-based no-gold-standard evaluation technique that accounts for correlated noise in measurements, enabling ranking of imaging methods without true reference values.

Findings

Derives a maximum-likelihood estimation method for correlated noise

Provides a framework to rank imaging methods based on measurement precision

Addresses limitations of previous methods assuming independent noise

Abstract

Objective evaluation of quantitative imaging (QI) methods using measurements directly obtained from patient images is highly desirable but hindered by the non-availability of gold standards. To address this issue, statistical techniques have been proposed to objectively evaluate QI methods without a gold standard. These techniques assume that the measured and true values are linearly related by a slope, bias, and normally distributed noise term, where it is assumed that the noise term between the different methods is independent. However, the noise could be correlated since it arises in the process of measuring the same true value. To address this issue, we propose a new no-gold-standard evaluation (NGSE) technique that models this noise as a multivariate normally distributed term, characterized by a covariance matrix. In this manuscript, we derive a maximum-likelihood-based technique that, without any knowledge of the true QI values, estimates the slope, bias, and covariance matrix terms. These are then used to rank the methods on the basis of precision of the measured QI values. Overall, the derivation demonstrates the mathematical premise behind the proposed NGSE technique.