Heterogeneous Treatment Effect with Trained Kernels of the Nadaraya-Watson Regression

TL;DR

This paper introduces TNW-CATE, a novel neural network-based method for estimating heterogeneous treatment effects by training kernels within a Nadaraya-Watson regression framework, especially effective when controls are abundant.

Contribution

The paper proposes a trainable kernel approach using neural subnetworks within Nadaraya-Watson regression for improved CATE estimation, leveraging transfer learning concepts.

Findings

TNW-CATE outperforms T-, S-, and X-learners in simulations.

The method effectively captures complex treatment effect heterogeneity.

Code implementation is publicly available for reproducibility.

Abstract

A new method for estimating the conditional average treatment effect is proposed in the paper. It is called TNW-CATE (the Trainable Nadaraya-Watson regression for CATE) and based on the assumption that the number of controls is rather large whereas the number of treatments is small. TNW-CATE uses the Nadaraya-Watson regression for predicting outcomes of patients from the control and treatment groups. The main idea behind TNW-CATE is to train kernels of the Nadaraya-Watson regression by using a weight sharing neural network of a specific form. The network is trained on controls, and it replaces standard kernels with a set of neural subnetworks with shared parameters such that every subnetwork implements the trainable kernel, but the whole network implements the Nadaraya-Watson estimator. The network memorizes how the feature vectors are located in the feature space. The proposed approach is similar to the transfer learning when domains of source and target data are similar, but tasks are different. Various numerical simulation experiments illustrate TNW-CATE and compare it with the well-known T-learner, S-learner and X-learner for several types of the control and treatment outcome functions. The code of proposed algorithms implementing TNW-CATE is available in https://github.com/Stasychbr/TNW-CATE.