Learning Optimal Policies from Observational Data

TL;DR

This paper addresses the challenge of learning optimal policies from observational data by providing theoretical bounds on estimation errors and proposing a domain adversarial neural network approach to mitigate selection bias and missing counterfactuals.

Contribution

It introduces a novel theoretical framework connecting counterfactual estimation errors to domain adaptation and develops a principled neural network method for policy optimization.

Findings

Theoretical bounds on counterfactual estimation errors are established.

Domain adversarial neural networks improve policy learning from observational data.

Experimental results demonstrate effectiveness on medical and UCI datasets.

Abstract

Choosing optimal (or at least better) policies is an important problem in domains from medicine to education to finance and many others. One approach to this problem is through controlled experiments/trials - but controlled experiments are expensive. Hence it is important to choose the best policies on the basis of observational data. This presents two difficult challenges: (i) missing counterfactuals, and (ii) selection bias. This paper presents theoretical bounds on estimation errors of counterfactuals from observational data by making connections to domain adaptation theory. It also presents a principled way of choosing optimal policies using domain adversarial neural networks. We illustrate the effectiveness of domain adversarial training together with various features of our algorithm on a semi-synthetic breast cancer dataset and a supervised UCI dataset (Statlog).