Stochastic Treatment Choice with Empirical Welfare Updating

TL;DR

This paper introduces a new stochastic approach for treatment assignment that updates a prior over policies using empirical welfare, employing variational Bayes for approximation, with proven convergence and application to real data.

Contribution

It develops a novel empirical welfare-based updating method for stochastic treatment rules using variational Bayes, with theoretical convergence guarantees.

Findings

Welfare regret converges at rate ln(n)/sqrt(n).

Method effectively applied to Job Training Partnership data.

Provides a practical framework for stochastic treatment policy optimization.

Abstract

This paper proposes a novel method to estimate individualised treatment assignment rules. The method is designed to find rules that are stochastic, reflecting uncertainty in estimation of an assignment rule and about its welfare performance. Our approach is to form a prior distribution over assignment rules, not over data generating processes, and to update this prior based upon an empirical welfare criterion, not likelihood. The social planner then assigns treatment by drawing a policy from the resulting posterior. We show analytically a welfare-optimal way of updating the prior using empirical welfare; this posterior is not feasible to compute, so we propose a variational Bayes approximation for the optimal posterior. We characterise the welfare regret convergence of the assignment rule based upon this variational Bayes approximation, showing that it converges to zero at a rate of ln(n)/sqrt(n). We apply our methods to experimental data from the Job Training Partnership Act Study to illustrate the implementation of our methods.