C-kNN-LSH: A Nearest-Neighbor Algorithm for Sequential Counterfactual Inference

TL;DR

This paper introduces C-kNN-LSH, a scalable nearest-neighbor method using locality-sensitive hashing for sequential causal inference in high-dimensional longitudinal data, with theoretical guarantees and superior real-world performance.

Contribution

We develop a novel C-kNN-LSH framework that combines locality-sensitive hashing with doubly-robust estimation for efficient, accurate causal inference in complex longitudinal datasets.

Findings

Demonstrates superior recovery heterogeneity estimation on Long COVID data.

Provides theoretical guarantees of estimator consistency and robustness.

Shows improved policy value estimation compared to existing methods.

Abstract

Estimating causal effects from longitudinal trajectories is central to understanding the progression of complex conditions and optimizing clinical decision-making, such as comorbidities and long COVID recovery. We introduce \emph{C-kNN--LSH}, a nearest-neighbor framework for sequential causal inference designed to handle such high-dimensional, confounded situations. By utilizing locality-sensitive hashing, we efficiently identify ``clinical twins'' with similar covariate histories, enabling local estimation of conditional treatment effects across evolving disease states. To mitigate bias from irregular sampling and shifting patient recovery profiles, we integrate neighborhood estimator with a doubly-robust correction. Theoretical analysis guarantees our estimator is consistent and second-order robust to nuisance error. Evaluated on a real-world Long COVID cohort with 13,511 participants, \emph{C-kNN-LSH} demonstrates superior performance in capturing recovery heterogeneity and estimating policy values compared to existing baselines.