An Experimental Evaluation of Regret-Based Econometrics

TL;DR

This paper evaluates regret-based econometrics using controlled ad-auction data, finding it as accurate as equilibrium-based methods and highlighting differences in human regret minimization across valuation types.

Contribution

It provides an experimental assessment of regret-based econometrics and explores human behavior in ad auctions, suggesting potential improvements for estimation accuracy.

Findings

Regret-based approach is as accurate as equilibrium-based methods.

High valuation players tend to minimize regret rationally.

Low valuation players overbid, showing bias in regret minimization.

Abstract

Using data obtained in a controlled ad-auction experiment that we ran, we evaluate the regret-based approach to econometrics that was recently suggested by Nekipelov, Syrgkanis, and Tardos (EC 2015). We found that despite the weak regret-based assumptions, the results were (at least) as accurate as those obtained using classic equilibrium-based assumptions. En route we studied to what extent humans actually minimize regret in our ad auction, and found a significant difference between the "high types" (players with a high valuation) who indeed rationally minimized regret and the "low types" who significantly overbid. We suggest that correcting for these biases and adjusting the regret-based econometric method may improve the accuracy of estimated values.