Optimal Bidding in Repeated Wireless Spectrum Auctions with Budget Constraints

TL;DR

This paper investigates optimal bidding strategies for budget-constrained operators in repeated wireless spectrum auctions, proposing a reinforcement learning approach that outperforms truthful bidding in terms of utility.

Contribution

It introduces a dynamic auction game model with reinforcement learning for distributed bidding under budget constraints in wireless spectrum markets.

Findings

Learning-based bidding yields higher utility than truthful bidding.

The Markov game model effectively captures opponent behavior.

Distributed algorithm performs well with limited local information.

Abstract

Small operators who take part in secondary wireless spectrum markets typically have strict budget limits. In this paper, we study the bidding problem of a budget constrained operator in repeated secondary spectrum auctions. In existing truthful auctions, truthful bidding is the optimal strategy of a bidder. However, budget limits impact bidding behaviors and make bidding decisions complicated, since bidders may behave differently to avoid running out of money. We formulate the problem as a dynamic auction game between operators, where knowledge of other operators is limited due to the distributed nature of wireless networks/markets. We first present a Markov Decision Process (MDP) formulation of the problem and characterize the optimal bidding strategy of an operator, provided that opponents' bids are i.i.d. Next, we generalize the formulation to a Markov game that, in conjunction with model-free reinforcement learning approaches, enables an operator to make inferences about its opponents based on local observations. Finally, we present a fully distributed learning-based bidding algorithm which relies only on local information. Our numerical results show that our proposed learning-based bidding results in a better utility than truthful bidding.