ADCB: An Alzheimer's disease benchmark for evaluating observational estimators of causal effects

TL;DR

This paper introduces ADCB, a sophisticated Alzheimer's disease simulator designed to evaluate causal effect estimators in healthcare, capturing real-world complexities and enabling benchmarking of various causal inference methods.

Contribution

The paper presents a novel Alzheimer's disease simulator grounded in real data, with adjustable parameters to test causal estimators under diverse, realistic conditions.

Findings

The simulator effectively models healthcare data complexities.

Different estimators' performances vary with simulation parameters.

The benchmark helps identify robust causal inference methods.

Abstract

Simulators make unique benchmarks for causal effect estimation since they do not rely on unverifiable assumptions or the ability to intervene on real-world systems, but are often too simple to capture important aspects of real applications. We propose a simulator of Alzheimer's disease aimed at modeling intricacies of healthcare data while enabling benchmarking of causal effect and policy estimators. We fit the system to the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset and ground hand-crafted components in results from comparative treatment trials and observational treatment patterns. The simulator includes parameters which alter the nature and difficulty of the causal inference tasks, such as latent variables, effect heterogeneity, length of observed history, behavior policy and sample size. We use the simulator to compare estimators of average and conditional treatment effects.