Pricing Experiments in Matching Marketplaces under Interference: Designs and Estimators

TL;DR

This paper investigates how to design and analyze pricing experiments in matching marketplaces considering interference, proposing estimators that reduce bias and validating their effectiveness through theoretical and numerical analysis.

Contribution

It introduces the shadow price estimator for pricing experiments under interference and analyzes the impact of design choices on bias in marketplace experiments.

Findings

Standard estimators are biased under interference.

The shadow price estimator reduces bias, especially when ignoring pricing differences.

Design choices significantly affect the bias direction and magnitude.

Abstract

Interference between treated and untreated units is a source of bias in marketplace experiments. In this paper, we specifically consider pricing interventions, in which a platform seeks to adjust base pricing levels at the marketplace level in order to increase demand. In a matching marketplace, this type of experiment leads to a crucial design question: should the platform match treated and untreated units differently because they paid different prices? We find that standard estimation techniques are biased, but the sign of this bias depends strongly on this design choice. Bias can be reduced by using the ``shadow price estimator'', which relies on the optimal dual solution of the platform's supply-demand matching problem -- especially when the platform chooses to ignore pricing differences at matching time. We validate our findings both theoretically in a fluid limit setting, and numerically in a finite-sample setting.