Interpretable Personalized Experimentation

TL;DR

This paper introduces a scalable, interpretable system for personalized experimentation that explains black-box heterogeneous treatment effect models and generates understandable personalized policies, deployed at Meta.

Contribution

It presents a novel system that makes black-box HTE models interpretable and deployable in production for personalized treatment policies at scale.

Findings

Effective explanation methods for black-box HTE models.

Successful deployment in Meta's production environment.

Insights from evaluation on public and Meta data.

Abstract

Black-box heterogeneous treatment effect (HTE) models are increasingly being used to create personalized policies that assign individuals to their optimal treatments. However, they are difficult to understand, and can be burdensome to maintain in a production environment. In this paper, we present a scalable, interpretable personalized experimentation system, implemented and deployed in production at Meta. The system works in a multiple treatment, multiple outcome setting typical at Meta to: (1) learn explanations for black-box HTE models; (2) generate interpretable personalized policies. We evaluate the methods used in the system on publicly available data and Meta use cases, and discuss lessons learnt during the development of the system.