Improving Variance Estimation for Covariate Adjustment with Binary Outcomes

TL;DR

This paper introduces an influence function-based leave-one-out variance estimator for covariate adjustment with binary outcomes, improving reliability in challenging scenarios like rare events or small samples.

Contribution

It proposes a novel variance estimator that offers better type-I error control and ease of implementation for binary outcomes in covariate-adjusted treatment effect estimation.

Findings

Estimator performs reliably in simulations with rare outcomes.

Provides appropriate type-I error control in small samples.

Offers a closed-form expression for straightforward implementation.

Abstract

Covariate adjustment is a general method for improving precision when estimating treatment effects in randomized trials and is recommended by the FDA in its 2023 guidance when baseline variables are prognostic for the primary outcome. We focus on a method highlighted in that guidance called ``standardization" (or ``g-computation") for estimating the marginal treatment effect. We address the question of how to reliably estimate variance for binary outcomes when marginal outcome probabilities are close to 0 or 1. We propose an influence function-based leave-one-out cross-validated (IF-LOO) variance estimator for the standardized difference-in-means average treatment effect. Through simulation studies, we show that this estimator provides appropriate type-I error control and performs reliably in challenging settings where existing methods can yield inflated type-I error or fail entirely, such as when outcome events are rare or sample sizes are small. In addition to having desirable statistical properties, we derive a closed-form expression for the proposed estimator, enabling straightforward and reliable implementation by study statisticians. The robust finite-sample performance and ease of implementation suggest the IF-LOO variance estimator is a prudent default choice for standardization in clinical trials.