Learning to Bid in Non-Stationary Repeated First-Price Auctions

TL;DR

This paper develops a learning framework for bidding strategies in non-stationary first-price auctions, introducing new metrics for opponent behavior regularity and achieving minimax-optimal dynamic regret bounds.

Contribution

It introduces two regularity metrics for opponent bids, characterizes minimax-optimal dynamic regret, and proposes an Optimistic Mirror Descent method tailored for non-stationary auction environments.

Findings

Our method outperforms existing algorithms on synthetic datasets.

Theoretical bounds match minimax lower bounds under regularity conditions.

New metrics effectively quantify non-stationarity in auction environments.

Abstract

First-price auctions have recently gained significant traction in digital advertising markets, exemplified by Google's transition from second-price to first-price auctions. Unlike in second-price auctions, where bidding one's private valuation is a dominant strategy, determining an optimal bidding strategy in first-price auctions is more complex. From a learning perspective, the learner (a specific bidder) can interact with the environment (other bidders, i.e., opponents) sequentially to infer their behaviors. Existing research often assumes specific environmental conditions and benchmarks performance against the best fixed policy (static benchmark). While this approach ensures strong learning guarantees, the static benchmark can deviate significantly from the optimal strategy in environments with even mild non-stationarity. To address such scenarios, a dynamic benchmark--representing the sum of the highest achievable rewards at each time step--offers a more suitable objective. However, achieving no-regret learning with respect to the dynamic benchmark requires additional constraints. By inspecting reward functions in online first-price auctions, we introduce two metrics to quantify the regularity of the sequence of opponents' highest bids, which serve as measures of non-stationarity. We provide a minimax-optimal characterization of the dynamic regret for the class of sequences of opponents' highest bids that satisfy either of these regularity constraints. Our main technical tool is the Optimistic Mirror Descent (OMD) framework with a novel optimism configuration, which is well-suited for achieving minimax-optimal dynamic regret rates in this context. We then use synthetic datasets to validate our theoretical guarantees and demonstrate that our methods outperform existing ones.