Constructing Confidence Intervals for Average Treatment Effects from Multiple Datasets

TL;DR

This paper introduces a new, assumption-light method for constructing valid confidence intervals for average treatment effects by combining multiple observational datasets, improving precision through prediction-powered inference.

Contribution

The paper presents a novel method for estimating ATE and constructing valid CIs from multiple datasets, including an extension to combine experimental and observational data.

Findings

The method provides unbiased ATE estimates with valid confidence intervals.

Numerical experiments confirm the theoretical properties of the proposed approach.

Extension allows combining experimental and observational datasets for CI construction.

Abstract

Constructing confidence intervals (CIs) for the average treatment effect (ATE) from patient records is crucial to assess the effectiveness and safety of drugs. However, patient records typically come from different hospitals, thus raising the question of how multiple observational datasets can be effectively combined for this purpose. In our paper, we propose a new method that estimates the ATE from multiple observational datasets and provides valid CIs. Our method makes little assumptions about the observational datasets and is thus widely applicable in medical practice. The key idea of our method is that we leverage prediction-powered inferences and thereby essentially `shrink' the CIs so that we offer more precise uncertainty quantification as compared to na\"ive approaches. We further prove the unbiasedness of our method and the validity of our CIs. We confirm our theoretical results through various numerical experiments. Finally, we provide an extension of our method for constructing CIs from combinations of experimental and observational datasets.