Test-negative designs with various reasons for testing: statistical bias and solution

TL;DR

This paper examines the bias introduced in test-negative vaccine effectiveness studies when including individuals tested for various reasons, and proposes stratification methods to correct it.

Contribution

It provides a formal statistical analysis of biases in modified test-negative designs and introduces stratified estimators to address these biases.

Findings

Standard odds ratio may be biased without accounting for reasons for testing

Stratification by reasons for testing can eliminate bias and improve estimates

Simulation studies demonstrate the effectiveness of the proposed stratified estimators

Abstract

Test-negative designs are widely used for post-market evaluation of vaccine effectiveness, particularly in cases when randomized trials are not feasible. Differing from classical test-negative designs where only healthcare-seekers with symptoms are included, recent test-negative designs have involved individuals with various reasons for testing, especially in an outbreak setting. While including these data can increase sample size and hence improve precision, concerns have been raised about whether they introduce bias into the current framework of test-negative designs, thereby demanding a formal statistical examination of this modified design. In this article, using statistical derivations, causal graphs, and numerical demonstrations, we show that the standard odds ratio estimator may be biased if various reasons for testing are not accounted for. To eliminate this bias, we identify three categories of reasons for testing, including symptoms, mandatory screening, and case contact tracing, and characterize associated statistical properties and estimands. Based on our characterization, we show how to consistently estimate each estimand via stratification. Furthermore, we describe when these estimands correspond to the same vaccine effectiveness parameter, and, when appropriate, propose a stratified estimator that can incorporate multiple reasons for testing and improve precision. The performance of our proposed method is demonstrated through simulation studies.