Causal inference methods for small non-randomized studies: Methods and an application in COVID-19

TL;DR

This paper reviews and compares various causal inference methods, especially propensity score techniques, for analyzing small non-randomized studies like early COVID-19 trials, highlighting their properties and applications.

Contribution

It provides a unified overview of alternative causal inference methods, including propensity scores and doubly robust estimators, with a focus on small sample settings in clinical research.

Findings

Propensity score methods help reduce bias in non-randomized studies.

Doubly robust estimators offer advantages in small samples.

Simulation results illustrate the properties of these methods.

Abstract

The usual development cycles are too slow for the development of vaccines, diagnostics and treatments in pandemics such as the ongoing SARS-CoV-2 pandemic. Given the pressure in such a situation, there is a risk that findings of early clinical trials are overinterpreted despite their limitations in terms of size and design. Motivated by a non-randomized open-label study investigating the efficacy of hydroxychloroquine in patients with COVID-19, we describe in a unified fashion various alternative approaches to the analysis of non-randomized studies. A widely used tool to reduce the impact of treatment-selection bias are so-called propensity score (PS) methods. Conditioning on the propensity score allows one to replicate the design of a randomized controlled trial, conditional on observed covariates. Extensions include the g-computation approach, which is less frequently applied, in particular in clinical studies. Moreover, doubly robust estimators provide additional advantages. Here, we investigate the properties of propensity score based methods including three variations of doubly robust estimators in small sample settings, typical for early trials, in a simulation study. R code for the simulations is provided.