Uplift vs. predictive modeling: a theoretical analysis

TL;DR

This paper provides a theoretical comparison between uplift and predictive modeling approaches, analyzing their performance based on various parameters and conditions, with implications for decision-making in multiple domains.

Contribution

It introduces a new profit measure, proves the convergence of the uplift curve, and identifies conditions where predictive models outperform uplift models.

Findings

Mutual information influences model performance

Uplift and predictive models converge under certain conditions

Predictive approaches can outperform uplift in specific scenarios

Abstract

Despite the growing popularity of machine-learning techniques in decision-making, the added value of causal-oriented strategies with respect to pure machine-learning approaches has rarely been quantified in the literature. These strategies are crucial for practitioners in various domains, such as marketing, telecommunications, health care and finance. This paper presents a comprehensive treatment of the subject, starting from firm theoretical foundations and highlighting the parameters that influence the performance of the uplift and predictive approaches. The focus of the paper is on a binary outcome case and a binary action, and the paper presents a theoretical analysis of uplift modeling, comparing it with the classical predictive approach. The main research contributions of the paper include a new formulation of the measure of profit, a formal proof of the convergence of the uplift curve to the measure of profit ,and an illustration, through simulations, of the conditions under which predictive approaches still outperform uplift modeling. We show that the mutual information between the features and the outcome plays a significant role, along with the variance of the estimators, the distribution of the potential outcomes and the underlying costs and benefits of the treatment and the outcome.