Bayesian Double Machine Learning for Causal Inference

TL;DR

This paper introduces a Bayesian Double Machine Learning approach for causal inference in high-dimensional models, addressing biases from machine learning methods and providing robust, efficient estimates with improved finite-sample properties.

Contribution

It develops a fully Bayesian method for causal inference that overcomes regularization bias and offers finite-sample robustness, extending the DML framework with a generative model.

Findings

Lower asymptotic bias compared to naive estimators

Achieves asymptotic normality and semiparametric efficiency

Outperforms existing methods in simulations with lower RMSE and better coverage

Abstract

This paper proposes a simple, novel, and fully-Bayesian approach for causal inference in partially linear models with high-dimensional control variables. Off-the-shelf machine learning methods can introduce biases in the causal parameter known as regularization-induced confounding. To address this, we propose a Bayesian Double Machine Learning (BDML) method, which modifies a standard Bayesian multivariate regression model and recovers the causal effect of interest from the reduced-form covariance matrix. Our BDML is related to the burgeoning frequentist literature on DML while addressing its limitations in finite-sample inference. Moreover, the BDML is based on a fully generative probability model in the DML context, adhering to the likelihood principle. We show that in high dimensional setups the naive estimator implicitly assumes no selection on observables--unlike our BDML. The BDML exhibits lower asymptotic bias and achieves asymptotic normality and semiparametric efficiency as established by a Bernstein-von Mises theorem, thereby ensuring robustness to misspecification. In simulations, our BDML achieves lower RMSE, better frequentist coverage, and shorter confidence interval width than alternatives from the literature, both Bayesian and frequentist.