Efficient and powerful equivalency test on combined mean and variance with application to diagnostic device comparison studies

TL;DR

This paper introduces a new statistical test for equivalency in medical device comparison studies, specifically addressing the combined mean and variance, with improved accuracy and controlled error rates.

Contribution

A novel generalized pivotal test for equivalency based on RMS, providing rigorous, accurate results with well-controlled type I error, especially useful for small to medium sample size studies.

Findings

The method has well-controlled type I error in simulations.

It performs favorably compared to existing methods.

Application to real oximetry data confirmed system equivalency.

Abstract

In medical device comparison studies, equivalency test is commonly used to demonstrate two measurement methods agree up to a pre-specified performance goal based on the paired repeated measures. Such equivalency test often involves controlling the absolute differences that depend on both the mean and variance parameters, and poses some challenges for statistical analysis. For example, for the oximetry comparison study that motivates our research, FDA has clear guidelines approving an investigational pulse oximeter in comparison to a standard oximeter via testing the root mean squares (RMS), a composite measure of both mean and variance parameters. For the hypothesis testing of this composite measure, existing methods have been either exploratory or relying on the large-sample normal approximation with conservative and unsatisfactory performance. We develop a novel generalized pivotal test to rigorously and accurately test the system equivalency based on RMS. The proposed method has well-controlled type I error and favorable performance in our extensive numerical studies. When analyzing data from an oximetry comparison study, aiming to demonstrate performance equivalency between an FDA-cleared oximetry system and an investigational system, our proposed method resulted in a highly significant test result strongly supporting the system equivalency. We also provide efficient R programs for the proposed method in a publicly available R package. Considering that many practical equivalency studies of diagnostic devices are of small to medium sizes, our proposed method and software timely bridge an existing gap in the field.