Learning Decision Policies with Instrumental Variables through Double Machine Learning

TL;DR

This paper introduces DML-IV, a novel non-linear instrumental variable regression method that reduces bias in two-stage models, enabling more accurate causal inference and policy learning in confounded data settings.

Contribution

The paper proposes DML-IV, a bias-reducing, double machine learning-based IV regression approach that improves causal effect estimation and policy learning in the presence of confounders.

Findings

DML-IV achieves strong convergence and $O(N^{-1/2})$ suboptimality guarantees.

DML-IV outperforms existing IV regression methods on benchmarks.

DML-IV learns high-performing policies despite confounding and instrumental variables.

Abstract

A common issue in learning decision-making policies in data-rich settings is spurious correlations in the offline dataset, which can be caused by hidden confounders. Instrumental variable (IV) regression, which utilises a key unconfounded variable known as the instrument, is a standard technique for learning causal relationships between confounded action, outcome, and context variables. Most recent IV regression algorithms use a two-stage approach, where a deep neural network (DNN) estimator learnt in the first stage is directly plugged into the second stage, in which another DNN is used to estimate the causal effect. Naively plugging the estimator can cause heavy bias in the second stage, especially when regularisation bias is present in the first stage estimator. We propose DML-IV, a non-linear IV regression method that reduces the bias in two-stage IV regressions and effectively learns high-performing policies. We derive a novel learning objective to reduce bias and design the DML-IV algorithm following the double/debiased machine learning (DML) framework. The learnt DML-IV estimator has strong convergence rate and $O(N^{-1/2})$ suboptimality guarantees that match those when the dataset is unconfounded. DML-IV outperforms state-of-the-art IV regression methods on IV regression benchmarks and learns high-performing policies in the presence of instruments.