Identifiable Energy-based Representations: An Application to Estimating Heterogeneous Causal Effects

TL;DR

This paper introduces an energy-based model that learns low-dimensional, identifiable representations of confounding variables to improve the estimation of heterogeneous causal effects, reducing sample complexity and enhancing accuracy.

Contribution

The paper proposes a novel energy-based model that produces partially identifiable, low-dimensional representations to facilitate more efficient and accurate CATE estimation, with theoretical guarantees.

Findings

Representations converge in experiments.

CATE estimation on learned representations outperforms other methods.

Model maintains partial identifiability and universal approximation capabilities.

Abstract

Conditional average treatment effects (CATEs) allow us to understand the effect heterogeneity across a large population of individuals. However, typical CATE learners assume all confounding variables are measured in order for the CATE to be identifiable. This requirement can be satisfied by collecting many variables, at the expense of increased sample complexity for estimating CATEs. To combat this, we propose an energy-based model (EBM) that learns a low-dimensional representation of the variables by employing a noise contrastive loss function. With our EBM we introduce a preprocessing step that alleviates the dimensionality curse for any existing learner developed for estimating CATEs. We prove that our EBM keeps the representations partially identifiable up to some universal constant, as well as having universal approximation capability. These properties enable the representations to converge and keep the CATE estimates consistent. Experiments demonstrate the convergence of the representations, as well as show that estimating CATEs on our representations performs better than on the variables or the representations obtained through other dimensionality reduction methods.