Beyond Demand Estimation: Consumer Surplus Evaluation via Cumulative Propensity Weights

TL;DR

This paper introduces a novel estimator for consumer surplus using observational data from randomized pricing, leveraging cumulative propensity weights and machine learning for robust, efficient evaluation of AI-driven decision impacts.

Contribution

It develops a new doubly robust estimator that avoids demand function estimation and integrates machine learning, improving accuracy and convergence in consumer surplus evaluation.

Findings

The proposed estimator accurately recovers consumer surplus in simulations.

It achieves fast convergence rates even with complex machine learning demand estimates.

The framework extends to fairness considerations, quantifying profit and equity trade-offs.

Abstract

This paper develops a practical framework for using observational data to audit the consumer surplus effects of AI-driven decisions, specifically in targeted pricing and algorithmic lending. Traditional approaches first estimate demand functions and then integrate to compute consumer surplus, but these methods can be challenging to implement in practice due to model misspecification in parametric demand forms and the large data requirements and slow convergence of flexible nonparametric or machine learning approaches. Instead, we exploit the randomness inherent in modern algorithmic pricing, arising from the need to balance exploration and exploitation, and introduce an estimator that avoids explicit estimation and numerical integration of the demand function. Each observed purchase outcome at a randomized price is an unbiased estimate of demand and by carefully reweighting purchase outcomes using novel cumulative propensity weights (CPW), we are able to reconstruct the integral. Building on this idea, we introduce a doubly robust variant named the augmented cumulative propensity weighting (ACPW) estimator that only requires one of either the demand model or the historical pricing policy distribution to be correctly specified. Furthermore, this approach facilitates the use of flexible machine learning methods for estimating consumer surplus, since it achieves fast convergence rates by incorporating an estimate of demand, even when the machine learning estimate has slower convergence rates. Neither of these estimators is a standard application of off-policy evaluation techniques as the target estimand, consumer surplus, is unobserved. To address fairness, we extend this framework to an inequality-aware surplus measure, allowing regulators and firms to quantify the profit-equity trade-off. Finally, we validate our methods through comprehensive numerical studies.