Price Experimentation and Interference
Ramesh Johari, Orrie B. Page, Gabriel Y. Weintraub

TL;DR
This paper investigates biases in price-related A/B experiments, revealing that standard estimators can mislead firms into decreasing profits, and proposes a debiasing method applicable with minimal experimental adjustments.
Contribution
It introduces a novel debiasing technique for price experiments that corrects sign errors in profit estimations using simple treatment-control splits.
Findings
Canonical estimators can have the wrong sign, leading to profit decreases.
Market factors influence the likelihood of sign errors in estimators.
Debiasing improves the reliability of price experiment results.
Abstract
In this paper, we examine the biases that arise when firms run A/B tests on continuous parameters to estimate global treatment effects on performance metrics of interest; we particularly focus on price experiments to measure the price impact on quantity demanded, and on profit. In canonical A/B experimental estimators, biases emerge due to interference between market participants. We employ structural modeling and differential calculus to derive intuitive characterizations of these biases. We then specialize our general model to the standard revenue-management pricing problem. This setting highlights a fundamental risk innate to A/B pricing experiments: that the canonical estimator for the expected change in profits, counterintuitively, can have the wrong sign in expectation. In other words, following the guidance of canonical estimators may lead firms to move prices (or fees) in the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Platforms and Economics · Consumer Market Behavior and Pricing · Auction Theory and Applications
