A pragmatic policy learning approach to account for users' fatigue in repeated auctions

TL;DR

This paper introduces a pragmatic policy learning approach for repeated auctions that accounts for user fatigue, improving bidding strategies by considering the long-term effects of current bids, leading to better business metrics.

Contribution

It proposes a new policy learning method that explicitly models user fatigue effects in repeated auctions, addressing the limitations of traditional myopic bidding strategies.

Findings

Mitigating impatience improves auction outcomes.

Counterfactual analysis shows long-term benefits.

Business metrics are significantly enhanced.

Abstract

Online advertising banners are sold in real-time through auctions.Typically, the more banners a user is shown, the smaller the marginalvalue of the next banner for this user is. This fact can be detected bybasic ML models, that can be used to predict how previously won auctionsdecrease the current opportunity value. However, learning is not enough toproduce a bid that correctly accounts for how winning the current auctionimpacts the future values. Indeed, a policy that uses this prediction tomaximize the expected payoff of the current auction could be dubbedimpatient because such policy does not fully account for the repeatednature of the auctions. Under this perspective, it seems that most biddersin the literature are impatient. Unsurprisingly, impatience induces a cost.We provide two empirical arguments for the importance of this cost ofimpatience. First, an offline counterfactual analysis and, second, a notablebusiness metrics improvement by mitigating the cost of impatience withpolicy learning