An exact adaptive test with superior design sensitivity in an observational study of treatments for ovarian cancer

TL;DR

This paper introduces an exact adaptive test designed to maximize sensitivity analysis power in observational studies, outperforming traditional tests like Wilcoxon's in detecting treatment effects under bias.

Contribution

It proposes a novel adaptive testing method that enhances design sensitivity and power in observational studies, especially when analyzing treatment effects with potential biases.

Findings

Brown's combined quantile average test has higher sensitivity power than Wilcoxon's.

Noether's test exhibits superior design sensitivity despite low Pitman efficiency.

The adaptive test outperforms standard tests in sensitivity analysis scenarios.

Abstract

A sensitivity analysis in an observational study determines the magnitude of bias from nonrandom treatment assignment that would need to be present to alter the qualitative conclusions of a na\"{\i}ve analysis that presumes all biases were removed by matching or by other analytic adjustments. The power of a sensitivity analysis and the design sensitivity anticipate the outcome of a sensitivity analysis under an assumed model for the generation of the data. It is known that the power of a sensitivity analysis is affected by the choice of test statistic, and, in particular, that a statistic with good Pitman efficiency in a randomized experiment, such as Wilcoxon's signed rank statistic, may have low power in a sensitivity analysis and low design sensitivity when compared to other statistics. For instance, for an additive treatment effect and errors that are Normal or logistic or $t$-distributed with 3 degrees of freedom, Brown's combined quantile average test has Pitman efficiency close to that of Wilcoxon's test but has higher power in a sensitivity analysis, while a version of Noether's test has poor Pitman efficiency in a randomized experiment but much higher design sensitivity so it is vastly more powerful than Wilcoxon's statistic in a sensitivity analysis if the sample size is sufficiently large.