Dynamic First Price Auctions Robust to Heterogeneous Buyers
Shipra Agrawal, Eric Balkanski, Vahab Mirrokni, Balasubramanian Sivan

TL;DR
This paper develops a dynamic auction mechanism that maintains near-optimal revenue despite the presence of diverse buyer behaviors, including myopic, forward-looking, and learning buyers, in a repeated auction setting.
Contribution
It introduces a simple state-based mechanism that is robust to heterogeneous buyer types and achieves a constant fraction of the optimal revenue.
Findings
Mechanism achieves a constant fraction of optimal revenue.
Robust to various buyer behaviors including learning and foresight.
Applicable to diverse, unknown buyer populations.
Abstract
We study dynamic mechanisms for optimizing revenue in repeated auctions, that are robust to heterogeneous forward-looking and learning behavior of the buyers. Typically it is assumed that the buyers are either all myopic or are all infinite lookahead, and that buyers understand and trust the mechanism. These assumptions raise the following question: is it possible to design approximately revenue optimal mechanisms when the buyer pool is heterogeneous? Facing a heterogeneous population of buyers with an unknown mixture of -lookahead buyers, myopic buyers, no-regret-learners and no-policy-regret learners, we design a simple state-based mechanism that achieves a constant fraction of the optimal achievable revenue.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAuction Theory and Applications · Advanced Bandit Algorithms Research · Consumer Market Behavior and Pricing
\declaretheorem
[name=Theorem, sibling=theorem]rThm \declaretheorem[name=Lemma, sibling=lemma]rLem \declaretheorem[name=Corollary, sibling=proposition]rCor \declaretheorem[name=Proposition, sibling=theorem]rPro
Dynamic First Price Auctions Robust to Heterogeneous Buyers
Shipra Agrawal111Columbia University. [email protected]. This research was supported in part by a Google Faculty Research Award and NSF CAREER award CMMI-1846792.
Eric Balkanski222Harvard University. [email protected]. This research was supported in part by a Google PhD Fellowship.
Vahab Mirrokni333Google Research, New York. [email protected].
Balasubramanian Sivan 444Google Research, New York. [email protected].
Abstract
We study dynamic mechanisms for optimizing revenue in repeated auctions, that are robust to heterogeneous forward-looking and learning behavior of the buyers. Typically it is assumed that the buyers are either all myopic or are all infinite lookahead, and that buyers understand and trust the mechanism. These assumptions raise the following question: is it possible to design approximately revenue optimal mechanisms when the buyer pool is heterogeneous? Facing a heterogeneous population of buyers with an unknown mixture of -lookahead buyers, myopic buyers, no-regret-learners and no-policy-regret learners, we design a simple state-based mechanism that achieves a constant fraction of the optimal achievable revenue.
1 Introduction
Bundling increases revenue in commerce: this is a well known fact widely used in practice (e.g., Amazon often suggests a “frequently bought together” bundle while browsing many items). This same principle is at the heart of why stateful repeated/dynamic auctions are significantly more lucrative than one-shot auctions: stateful dynamic auctions profitably use the opportunity to bundle across time. This revenue opportunity has inspired a series of recent works [PPPR16, ADH16, MLTZ16, BML17, LP17, MLTZ18, ADMS18, BMSW18, BMLZ19] on designing dynamic auctions satisfying various desired properties.
The ability to link auctions across time significantly expands the design space of auctions allowing some exceedingly complex auctions. How does a buyer optimize when bidding in such auctions? The standard and widely used notion in the literature of incentive compatibility (IC) requires that the buyers understand these complicated auctions well, and have an “infinite lookahead”. I.e., they require the buyer to think about the consequences of their bid in the current round on all future round utilities and optimize current round bid accordingly. In particular, as [ADMS18] note, this assumes that the buyers understand the mechanism deeply enough to optimally respond, they believe that their interactions with the seller will last for all future rounds accounted for when computing utility, believe that the seller is indeed strictly following the advertised mechanism etc.
Numerous practical reasons make these assumptions far from true in reality: buyer’s computational limitations, inability to predict the future well enough, inability to trust a seller or understand the exact auction that is run in a complex supply chain of auctions. A striking example of this is the display ads market in Internet advertising. Given the number of ad exchanges, and the variety of purchase mechanisms that are constantly evolving over time, buyers are often unable to trust or verify whether a seller has stuck to an announced mechanism.
The consequence is that the seller faces a heterogeneous buyer population employing a large spectrum of strategies to maximize their perceived utility. As [ADMS18] note, such a buyer could behave myopically, or have a limited lookahead (i.e., a -lookahead instead of infinite lookahead), or be a learner that makes decisions only based on past performance of various bidding strategies thereby completely disregarding the seller’s description/promises in the mechanism’s future.
The solution to tackling a heterogeneous buyer behavior cannot be a buyer-specific auction that tailors the optimal auction for a given buyer behavior. Implementing such a discriminative auction may be legally infeasible and also impractical (buyer’s behavior may even change over time). Motivated by these observations, [ADMS18] considered the setting of a single seller repeatedly interacting with a single buyer whose behavior (myopic/infinite lookahead/learner etc.) the seller is a priori unaware of, and designed a single mechanism that simultaneously obtains a constant fraction of the optimal revenue achievable against each potential buyer type. In this paper, we study the following question:
Is there an -buyer mechanism that is robust against heterogeneous buyer behaviors?
We consider a general setting with an arbitrary and unknown mixture of multiple heterogeneous buyers. The challenges introduced by considering multiple buyers are discussed after describing the setting and main result.
The setting.
We study a repeated interaction between a single seller and buyers over rounds. At the beginning of each round , there is a single fresh good for sale whose private value for buyer is drawn, independently from other and , from a publicly known distribution with finite expectation . The buyer observes the valuation before making a bid . The good for sale in round has to be either allocated to one of the buyers or discarded immediately. Each buyer’s valuation is additive across rounds. We consider a range of buyer behaviors similar to [ADMS18], including myopic buyers, -lookahead buyers, no-regret learners and no-policy-regret learners. Formal definitions of these different buyer behaviors are provided in Section 2.
We categorize as sophisticated buyers the buyers who are either -lookahead for or no-policy-regret learners, and naive buyers those who are myopic ([math]-lookahead) or no-regret learners. Given a population with sophisticated and naive buyers, a robust mechanism aims to achieve close to maximum per-round revenue achievable from such a population, without prior knowledge of which buyer is naive or sophisticated, or the values of and .
Upper bound on revenue.
With only naive buyers, e.g. myopic buyers, it is impossible in each round to get more than the optimal revenue in a single-round auction obtainable from Myerson’s auction [Mye81a]. With only sophisticated buyers, it is impossible to get more than revenue, as revenue is upper bounded by the maximum value. It is easy to show that (see Appendix A). In a setting with and buyers, the total revenue achievable is thus at most (see Appendix A). Motivated by this upper bound, we define the following benchmark.
Definition 1**.**
We call a mechanism -robust* if, for any per-round valuation distribution , and for every value of (and ), without knowing or , it achieves an expected per round revenue of at least *
Main result.
We construct a mechanism that is Interim Individually Rational (IIR)555[ADMS18] show that it is impossible to achieve such a high-revenue with per-round ex-post IR unless the buyer’s lookahead is very high even in a single buyer setting, which rules that out as well in our setting. and -robust. I.e., for every value of , without knowing or , it achieves a per-round revenue within a constant factor of the near optimal revenue. The mechanism is a simple-to-implement first price auction with reserve price based on the current state of buyers (where state is a succinct summary of the buyer’s history). With the display ads industry moving to use first-price auctions [ada, dig, ade], this result is especially significant.
1.1 Overview of challenges and technical approach
The interactions between buyers, with heterogeneous and unknown lookahead and learning behaviors, introduce multiple challenges compared to the single buyer setting studied in [ADMS18].
Firstly an equilibrium, which is a profile of mutually best responding strategies from all agents, and a widely used predictor of a mechanism’s outcome, is unlikely to exist in our setting. Therefore it is not possible to prove revenue guarantees by arguing about what revenue would be obtained in the equilibrium outcome. Indeed, far from being able to pinpoint what our mechanism’s outcome will be, we only guarantee what the mechanism’s outcome will not be, as long as all agents satisfy much weaker notions of rationality like playing undominated strategies [BLP06] or no-regret strategies. This aligns well with our assumptions of heterogeneity and distrust among buyers. With just this guarantee of which outcomes will not occur in our mechanism, we are able to establish our strong revenue guarantees. In contrast, in the 1-buyer setting of [ADMS18], there are no equilibrium concerns: the single buyer simply best responds according to his utility function.
Another non-trivial challenge is that while proving results in undominated strategies, we have to establish a given strategy for buyer being dominated regardless of another buyer ’s strategies. In particular ’s strategy could be an arbitrary function of the entire history of not only ’s own bids and outcomes, but also those of . This creates complex dependencies of buyer ’s future utility on his bid today. For example, even if bids truthfully according to his valuations today, he has to consider a strategy of that would bid very high and sabotage in the future if it sees bidding beyond a certain bid in the current round. It requires careful design choices in the mechanism to be able to guarantee certain buyer behavior under such complex side-effects of a buyer’s bid. Key aspects of our mechanism are designed to allow lower bounding the utility of a buyer irrespective of other buyers’ behavior. This ability is crucial for establishing our revenue guarantees.
We construct a state-based mechanism where sophisticated buyers remain in a good state yielding high revenue, and where naive buyers bid as in an approximately revenue optimal one round auction. The mechanism incentivizes sophisticated buyers to remain in this good state in spite of the arbitrariness of other buyers’ responses, while also achieving high revenue. To guarantee high utility to every sophisticated buyer in the good state, the mechanism temporarily “rests” buyers who have recently been allocated a large number of items. Ignoring such buyers for a small number of rounds guarantees some rounds for each buyer to enjoy positive utility and get the item as long as they bid high enough. To do well against the benchmark of that extracts the optimal revenue possible from every buyer category, the mechanism necessarily has to adapt to the mixture of buyer population it is observing. Incorporating this adaptivity, and handling the resulting complications in utility analysis are some further challenges that we handle in our mechanism design and analysis.
1.2 Related work
There are several streams of literature related to our work in dynamic mechanism design.
Revenue maximization in dynamic auctions.
The stream closest to our paper is the work on revenue maximization in repeated auctions [PPPR16, ADH16, MLTZ16, BML17, LP17, MLTZ18, ADMS18, BMSW18, BMLZ19]. The main difference of our work from all of these, with the exception of [ADMS18, BMSW18], is the notion of dynamic incentive compatibility which assumes that all buyers are infinite lookahead, while we allow arbitrary mixtures of buyer attitudes. Agrawal et al. [ADMS18] study the -buyer setting and design robust auctions when the buyers are -lookahead buyers or no-simple-regret learners or no-policy-regret learners. Braverman et al. [BMSW18] also study the -buyer setting and design mechanisms to extract more than Myerson’s revenue when the buyers follow no-simple-regret learning strategies, and in particular a subset of them called mean-based bidding strategies.
Repeated interactions with evolving values.
There is a large body of work on designing mechanisms in repeated interactions when the buyers’ values evolve over time. See [BB84, Bes85, CH00, Bat05, ES07, AS13, KLN13, PST14, BS15, CDKS16] for an overview of dynamic mechanisms in such settings.
Bargaining, durable goods monopolist and Coase conjecture.
Unlike our setting where the value is drawn independently in every round, there is a large body of literature in economics that studies settings where the value is initially drawn from a distribution but remains fixed later on. This setting can be motivated based on several applications including bargaining, durable goods monopoly and behavior based discrimination. See [FVB06] for an excellent survey and references therein for an overview of this area and [DPS15, ILPT17] for work in the theoretical computer science literature.
Lookahead search.
The study of -lookahead search can be viewed in the context of bounded rationality, as pioneered by Herb Simon [Sim55]. He argued that, instead of optimizing, agents may apply a class of heuristics, termed satisfying heuristics in decision making. A natural choice of such heuristics is restricting the search space of best-response moves. Lookahead search in decision-making has been motivated and examined in great extent by the artificial intelligence community [Nau83, dKaS92, SKN09]. Lookahead search is also related to the sequential thinking framework in game theory [SW94]. More recently, [MTV12] study the quality of equilibrium outcomes for look-ahead search strategies for various classes of games. They observe that the quality of resulting equilibria increases in generalized second-price auctions, and duopoly games, but not in other classes of games. No prior work studies dynamic mechanisms that are robust against various lookahead search strategies.
2 Preliminaries
There are buyers and a single item for auction at each of sequential time steps. The value of buyer at time step for the item is denoted by and is drawn i.i.d. from a common prior distribution . The distribution is known to all the buyers and the seller. The realization of the value is private and is visible only to buyer . In each round , every buyer makes a bid . A buyer may use history of bids, allocation, and payments before round along with its own private valuations until round , to decide the bid in round . The seller uses the entire bid profile at time , along with history (bids, allocations, and payments) before time to decide allocation and payment for each buyer, such that .
More precisely, the bidding, payment, and allocation strategy space are defined as follows.
Definition 2**.**
Let denote the history of bids, allocation, and payment before time .
Definition 3** (Bidding strategy).**
Buyer ’s bids in round are decided by functions , i.e. bids in round are (possibly randomized) functions of history and buyer’s private valuation in round .
Definition 4** (Payment and allocation function).**
For each buyer , payment and allocation in round are given by and of history and bid profile in round . Here, denotes the probability of allocation to bidder , and denotes the bidder’s expected payment.
Definition 5** (Buyer’s realized utility).**
Buyer ’s realized utility in a round is given by , i.e., it is the difference between valuation and payment if the good is allocated to the buyer, and [math] otherwise. We often refer to this as simply the buyer’s utility in a round.
Remark 1**.**
Our current definition of history includes everything from past that can possibly be revealed to buyers and can be relevant to a buyer’s bidding strategy. However, our results are not tied to this particular definition of history. For example, instead of revealing the entire vector of bids from the past, if we happen to just reveal the allocation and payment after each auction, that would simply restrict the buyer’s strategy space further. Our arguments will hold as long as a buyer can see their own bids, allocation, and payments in the past, and whether or not the good was allocated in each round.
Remark 2**.**
Technically, the history for buyer should also include their own private valuations in the past. However, since the valuations are generated independently in each round from a known valuation distribution, are visible only to the buyer, and the buyer’s total utility is additive across rounds, the past valuations are irrelevant for the buyer’s bidding strategy. Therefore, for simplicity, we chose to eliminate them from the definition of buyer’s history.
Also define buyer i’s history, . would things should be still fine: this is because whatever be the strategies other buyers use, in a domination argument, we are going to fix those strategies, and then show that a chosen strategy for a protagonist buyer is dominated by another strategy.
In this paper, we focus on the first price auction mechanism which is defined by the following specific allocation and payment function, with flexibility to choose the mechanism for setting a “reserve price”.
Definition 6** (First price auction with reserve price).**
In a first price auction of a single good with reserve price , the seller observes the bids of participating buyers, and then allocates the good to the buyer with highest bid if that bid is above or equal to the reserve price. The winning buyer’s payment is equal to their bid.
2.1 Heterogeneous lookahead behavior
We define heterogeneous forward-looking behavior of buyers by considering buyers who may be myopic or -lookahead for different values of . A myopic (-lookahead) buyer is defined as a buyer who optimizes her myopic (-lookahead) utility in every round to decide the bid. Below, we give precise definitions of these. Intuitively, myopic buyers optimize their current round utility, while -lookahead buyers () optimize their total expected utility over the current and next rounds.
Definition 7** (Buyer’s myopic utility).**
Under bidding strategies , the myopic utility of buyer in round , given private valuation and history , is defined as
[TABLE]
Definition 8** (-lookahead utility).**
*Under bidding strategies , the -lookahead utility of buyer in round , given private valuation and history , is defined as *
[TABLE]
where denotes the vector of realized bids, allocation and payments in round , . And denotes the realizations of private valuations for all bidders from time to , i.e., .
Undominated strategies.
We provide revenue guarantees for our mechanism under an assumption that all myopic or -lookahead buyers play “undominated strategies”. This is a significantly more robust notion of rationality than a Nash equilibrium.
Definition 9** (Dominated strategies for -lookahead buyers.).**
In our setting for -lookahead buyer, a strategy is a dominated strategy at time under history if such that
[TABLE]
[TABLE]
For myopic buyers, above definition applies with .
2.2 Heterogeneous learning behavior
We consider learning buyers as those who do not know (or do not trust) the seller’s mechanism, in particular the seller’s allocation and payment function, in order to be able to precisely evaluate their current and future utility. Instead, a learning buyer uses the past outcomes to learn how to bid. We formalize the notion of learning buyer using the experts learning framework [FS95].
A learning buyer uses a learning algorithm in order to learn to bid in a way that its total utility is close to that achieved by the best single expert among a set of expert bidding strategies . (Recall from Definition 3, a bidding strategy is an arbitrary mapping from history and valuation to bid). We formalize different levels of learning sophistication among buyers by considering two classes of learning algorithms, as described below.
No-regret learner:
A no-regret learning buyer uses a no-regret learning algorithm to decide bid at time . The ‘reward’ (in no-regret learning terminology) at time on making a bid is given by the buyer’s th-round utility, determined by the mechanism’s output depending on other buyers’ bids as well as the history. That is, on making a bid , the learner’s reward at time is given by a function defined as
[TABLE]
Regret is defined as the difference between buyer’s total reward and that of the best expert in hindsight:
[TABLE]
A no-regret learning buyer uses a bidding strategy such the above regret is under every trajectory of bids and private valuations. Note that such a learner is solving an adversarial bandit problem, since the learner only observes the value of function on the bid used by the buyer. When the number of experts is finite, there are efficient and natural algorithms (e.g., EXP3 algorithm based on multiplicative weight updates [ACBFS03]) that achieve regret.
No-policy-regret learner:
This more sophisticated buyer uses a no-policy-regret learning algorithm (following definition of policy regret in [ADT12]). An important distinction from the definition of regret in the previous paragraph is that now the total reward of the best expert must be evaluated over the trajectory of adversarial inputs (i.e., history and other buyers’ bids) in response to the bids made by the expert. To make explicit the dependence of utility reward on the trajectory of past decisions through history of outcomes and other buyers’ strategic response, let us denote the reward function for round as . Let denote the strategy used by buyer at time and is the private valuation of buyer . Then, the learner’s expected reward in round is defined as:
[TABLE]
Then, for any sequence of other buyers’ valuations and strategies for policy-regret of such a buyer is defined against the best expert in hindsight:
[TABLE]
where denotes the bid made by buyer at time , denotes the bid that would be made by the expert under counterfactual trajectory, denotes the (possibly randomized) counterfactual trajectory of history that would be observed in response to using the bids suggested by the expert, instead of the original bids . A no-policy-regret learning buyer uses a bidding strategy such that the above policy-regret is guaranteed to be under any sequence of other buyers’ valuations and strategies for . See Appendix F for a short note on the existence of policy regret learning algorithms.
3 Repeated First Price Auction Mechanism
Algorithms 1-3 contain the formal description of our mechanism. We design a dynamic first price auction mechanism that is conducted in sequential rounds . In every round, the mechanism partitions the buyers into two categories: good state buyers and bad state buyers. All buyers start in good state. The buyers may be moved by the mechanism from good state to bad state over time but once in bad state, a buyer remains there for the remaining rounds.666One can think of the time period as the number of auctions in a day, and reset all buyers to good state at the beginning of the next day when another auctions are run. Alternatively it also possible to design a mechanism that permits the movement back to a good state for initial few rounds, but for clarity in exposition, so we choose to not do that here. In any round, the current set of buyers in good state and bad state are denoted by and respectively. The mechanism uses these states to track buyer behavior and incentivize lookahead or learning buyers to stay in a good state, in order to extract the desired revenue from both sophisticated and naive buyers.
The mechanism proceeds in epochs, each consisting of multiple rounds. In a given round, the mechanism either conducts a first price auction with a reserve price among the good state buyers, or a first price auction with a (different) reserve price among the bad state buyers. Specifically, an epoch consists of rounds where and is the number of good state buyers in the beginning of the epoch. During an epoch, the mechanism first runs the Bad-State-Auctions subroutine (Algorithm 2), which consists of rounds of auctions among bad state buyers. It then runs the Good-State-Auctions subroutine (Algorithm 3), which consists of rounds of auctions among good state buyers. Some good state buyers may be moved to bad state during the Good-State-Auctions. The reserve prices and for good and bad state auctions depend on parameters which are set at the beginning of an epoch, and remain fixed throughout an epoch.
In the first rounds of the epoch, Bad-State-Auctions runs a first price auction with reserve price among bad state buyers in , as described in Algorithm 2. Here , with , being the probability that a buyer wins in a Myerson auction with buyers (e.g., see [Mye81b]); and the quantile. In fact, for any , , so that and .
In each of the remaining rounds of the epoch, the Good-State-Auctions subroutine (Algorithm 3) runs a first price auction with reserve price among buyers currently in good state (i.e., buyers in ). Here, , with .
The Good-State-Auctions subroutine uses a third state, called the rest state, which is used to temporarily rest a buyer, i.e., not allow that buyer to participate in the remaining auctions in that epoch. The set of buyers in the rest state in the current round is denoted by . In each round, after the auction a buyer may be moved from good state to either bad or rest state in the following ways:
- •
The mechanism considers the number of uncleared auctions so far in this epoch, i.e., the number of auctions during this instance of Good-State-Auctions, where all the participating bids were lower than reserve price. If this number is greater than or equal to , then any good state buyer () whose bid in this round was smaller than the reserve price, is moved to bad state.
- •
For every buyer currently in good state (), the mechanism considers the number of allocations received by the buyer so far in this epoch. If this is , the buyer is moved to the rest state.
In the first step, the mechanism checks if the number of uncleared auctions is significantly above the statistically expected number. And, if so, from there on, the mechanism punishes every participating buyer who bids below reserve price. This step is aimed to ensure that the lookahead buyers are incentivized to bid above reserve price in rounds where their private valuations are high enough. To understand the intuition behind the second step, observe that given the epoch length of , statistically, any given buyer is expected to have the highest valuation among buyers for roughly a constant number of steps in every epoch. Thus, the second step is intended to ensure that a buyer does not win too many auctions (perhaps at cost of negative immediate utility for some rounds) in order to deprive other buyers of allocations and potentially cause the mechanism to move them to bad state.
4 Revenue Analysis: Main Result
Our main result is that the mechanism presented in the previous section extracts a constant factor of optimal revenue from sophisticated buyers and naive buyers. To formally state this result, we first define the sophisticated buyers and naive buyers. This involves defining the set of experts used by learning buyers. In general, an expert bidding strategy can be any arbitrary mapping from historical information and current valuation to bid. However, being able to learn the best strategy in such an arbitrary set makes too strong an assumption on learning abilities of the buyer. In fact, it is sufficient for our mechanism to have learning buyers that can compete against a restricted set of experts, as defined below.
Definition 10** (Expert set ).**
Let be a fixed size projection of history containing the following information: the buyer ’s state (whether it is bad or good/rest) in round , the number of buyers in good and bad state, and the number of uncleared auctions so far in the current epoch. Let denote the set of () possible values of . Then, set of experts is defined as mappings from this projected history and current valuation to a bid, i.e.,
Here, is a discretized (to arbitrary accuracy) range of valuations, in order to obtain a finite set of experts. Given , and valuation , an expert strategy suggests bid to buyer in round .
We are now ready to formally define sophisticated and naive buyers.
Definition 11** (Sophisticated and Naive buyers).**
Sophisticated buyers are defined as the buyers who are either -lookahead, for , or no-policy-regret learners against some set of experts containing . Naïve buyers are defined as buyers who are either myopic, or are no-regret learners against some set of experts containing .
{rThm}
[]
Assuming all myopic and -lookahead buyers play undominated strategies, the expected per round revenue of the mechanism described in Algorithm 1-3, with , , , is at least
[TABLE]
where is the number of sophisticated and naive buyers, respectively. More precisely, the expected per round revenue is at least
[TABLE]
where , with being the quantile for the valuation distribution, and . is the optimal revenue in a single-item auction with buyers.
The proof of Theorem 11 consists of four parts. We first give revenue and utility bounds that apply to any buyer, then characterize undominated strategies for myopic/lookahead buyers, and no-regret strategies for learners, and finally combine these parts. We give an overview of each part here and defer lemma statements and their proofs to the appendix.
General revenue and utility analysis (Appendix B).
We first establish in Lemma 1 that, if is the set of good state buyers at the end of an epoch, then the expected revenue from Good-State-Auctions during that epoch is at least . This lower bound on revenue from good state buyers is obtained by observing that each buyer who ends an epoch in good state must have either been allocated the item (and paid at least ) during at least rounds of this epoch to be moved to a rest state, or must have bid at least at each round where . In Lemma 2, we show that if the bid of bad state buyers is at least when their value is larger than , then the expected revenue per round of Bad-State-Auctions is at least .
A main part of the overall revenue analysis is to argue that sophisticated buyers are incentivized to remain in good state, irrespectively of other buyers’ bids. To show this, we establish a lower bound on the utility achievable in good state and an upper bound on the utility achievable in bad state. To establish the lower bound, we analyze a strategy called the good strategy (Definition 12) that never moves a buyer to the bad state. When , bids if , and [math] otherwise. When , bids . In Lemma 12, we lower bound the expected utility obtained by strategy over an epoch. The crucial component of the mechanism which allows this bound is temporarily moving buyers who have already been allocated enough (more than statistically expected) number of items to the rest state. This temporarily removes such buyers from good state auctions, and guarantees to any buyer a minimum number of rounds in each epoch where can get the item if it bids above the reserve price. In Lemma 3, we upper bound the expected utility achievable by any bad state buyer.
Undominated strategies for buyers with heterogeneous lookahead attitudes (Appendix C).
The main lemma for this part (Lemma 4) shows that a -lookahead buyer, for large enough, never enters the bad state. To show this, we consider a round where a buyer faces the threat to be sent to a bad state if it bids below . In Lemma 6, we lower bound the -lookahead utility obtained by strategy , which maintains in good state, in such a round. Lemma 5 then upper bounds the -lookahead utility of any strategy bidding below in such a round, which would send the buyer to a bad state. Lemma 4 then combines these two bounds to show that any strategy that sends a -lookahead buyer to a bad state is dominated by strategy . A main difficulty in combining these two lemmas is that the epoch lengths and the reserve prices vary at each epoch, and we need to compare utilities from different epochs. We show in Lemma 8 that a myopic buyer bids at least the reserve price when it has value at least in bad state.
Strategies of no-regret buyers with heterogeneous learning behaviors (Appendix D).
We show in Lemma 9 that a buyer that goes to a bad state has high policy-regret compared to an expert that plays strategy , which implies that a no-policy regret learner must remain in good state in all but rounds. A difficulty here is to argue that there is gap between the utility a buyer going to a bad state and the utility of an expert following the good strategy, where the utilities are evaluated over different trajectories of adversarial inputs. In Lemma 10, we give a condition under which a no-regret learner in bad state must bid at least the reserve price when its value is larger than in all but rounds. An important subtlety for no-regret learners is that due to the other buyers, a learner is not guaranteed to win a bad state auction and obtain positive utility when it bids at least .
Main result (Appendix E).
We combine the three previous parts to lower bound the revenue achieved by the mechanism and obtain Theorem 11. A last non-trivial argument needed is that if a naive buyer remains in good state, we obtain at least a much revenue from that buyer as if it was in bad state, regardless of how many buyers are in good and bad state.
Appendix A Proof for Revenue Upper Bound
Upper bound on revenue.
In a setting with just naive buyers and no other buyers (for e.g., just myopic buyers), it is impossible to get more than per round, i.e., the optimal revenue in a single-round auction obtainable from Myerson’s auction [Mye81a]. In a setting with just sophisticated buyers (and no other buyers), it is impossible to get more than as revenue is upper bounded by the maximum valuation. It is easy to show that . To see this, note that because a buyer has the largest value among buyers (with ties broken uniformly) with probability ,
[TABLE]
Now, since the expected value of a buyer conditioned on an event happening with probability is at most , the inequality follows.
Now, combining these, we claim that in a setting with sophisticated buyers and naive buyers, the total revenue achievable is at most . Indeed, if we were able to achieve more than this, then either the revenue contribution from the sophisticated buyers is more than or the naive buyers is more than — neither of this is possible because if that was true then in a setting with just the sophisticated buyers or just the naive buyers we could have simulated the rest of the buyers by adding dummy buyers and discarded the revenue contributed by dummy buyers to obtain more revenue than or .
Appendix B General Revenue and Utility Analysis
The analysis shows that, up to constant factors, the mechanism extracts the optimal revenue from sophisticated buyers and naïve buyers. In Section B.1, we first provide separate bounds on the revenue from good and bad state buyers in Lemma 1 and Lemma 2 respectively.
The main part of the analysis is to argue that sophisticated buyers, either -lookahead buyers for large enough or no-policy-regret learners, are incentivized to remain in the good state. We show that any strategy which leads to the bad state is a dominated strategy for -lookahead buyers (and has large regret for no-policy regret learners). To show this, in Section B.2, we provide lower and upper bounds on the utility achievable by a buyer in the good and bad state in Lemma 12 and Lemma 3, respectively.
These utility bounds are used Section C and Section D to argue that (a) -lookahead buyers for large enough (Lemma 4) and no-policy-regret buyers (Lemma 9) have incentive to stay in good state for most rounds, and (b) in bad state, myopic buyers (Lemma 8) and learning buyers (Lemma 10) have incentive to bid above reserve price when their private valuation is large enough.
Finally in Section E, we combine all these observations to lower bound the revenue achieved given a pool of heterogeneous lookahead and learning buyers, and prove our main result.
B.1 Revenue analysis from good and bad state auctions
We give bounds on the revenue achieved by Good-State-Auctions and Bad-State-Auctions. These are general bounds which hold for both lookahead and learning buyers.
Lemma 1**.**
Let be the set of good state buyers at the end of an epoch. Then, total expected revenue from Good-State-Auctions during that epoch is at least .
Proof.
Let be the good state buyers at the end of an epoch of the mechanism. Consider the round during that epoch at which the number of uncleared auctions reaches . Such a step must exist because total number of allocations is at most ( is the number of good state buyers at the beginning of that epoch) but the number of good state auctions in this epoch is . Suppose that at this round, some buyer in is in good state (i.e., hasn’t yet been moved to rest state). Every such buyer must bid or above in each of the remaining rounds of the epoch, until that buyer is moved to the rest state; otherwise, the mechanism would have pushed this buyer to a bad state and this buyer would not be in at the end of this epoch. This means that before the end of the epoch: either all of the buyers in were moved to rest state so that revenue was at least ; or all the remaining auctions (after the number of uncleared auctions reached the threshold) cleared because some bid exceeded reserve price, so that the number of uncleared auctions is bounded by the threshold , and the number of goods sold through the good state auctions is at least , giving revenue of at least . ∎
Revenue from bad state auctions.
Lemma 2**.**
*Let be set of bad state buyers in the beginning of an epoch. Suppose that in every round of Bad-State-Auctions during this epoch where the set of buyers with is non-empty, at least one such buyer is guaranteed to bid or more. Then, total expected revenue from that epoch is at least , where expectation is taken over valuations . *
Proof.
Recall . If in every round, among buyers in bad state with valuation above , at least one buyer is guaranteed to bid above , then the mechanism will get at least fraction of the expected revenue of a posted-price mechanism with a uniform price among buyers in . Since is the probability of a buyer winning an buyer Myerson’s auction with iid values, where , this revenue (see for example [CHMS10]) is at least for every round in bad state auctions. ∎
B.2 Utility analysis from good and bad state buyers
We give bounds on the utility achievable by a buyer in a good state and bad state. These are general bounds which hold both for buyers with lookahead attitudes and the buyers with learning behaviors. Further, they hold irrespective of other buyers’ bids.
Lower bound on the utility achievable by a good state buyer.
We lower bound the utility achievable by a buyer in the good state by describing a simple strategy which (1) never moves a buyer to the bad state and (2) achieves high utility. We call this strategy the ‘good strategy’, denoted by , and defined as follows.
Definition 12** (Good strategy ).**
In any round of Good-State-Auctions, a buyer using good strategy bids in the following manner. If the number of the uncleared past auctions in the current epoch is , bid if the current valuation , and bid [math], otherwise. If , set the bid equal to the reserve price irrespective of the current valuation.
Observe that the good strategy is defined in such a manner that a buyer using this strategy for all rounds of Good-State-Auctions is guaranteed to either always stay in the good state or be moved to the rest state, i.e., is guaranteed to be never pushed to a bad state. Next, we now lower bound the utility of a buyer using this strategy.
{rLem}
[]
Consider a buyer in good state at the beginning of an epoch. Irrespective of the other buyers’ bids, the expected utility of strategy over that epoch is at least
[TABLE]
Proof.
Consider an epoch where a good state buyer follows strategy for all rounds in Good-State-Auctions. Similarly to the proof of Lemma 1, observe that the number of uncleared auction in such an epoch must reach before the end of this epoch. Let denote the event that the buyer is still in good state when the number of uncleared auction in that epoch reaches , i.e., the buyer did not get enough allocations to go to rest state. Since the buyer bids whenever , probability of this event happening is bounded by the probability of the following event: let be independent samples from distribution ; then consider event . Since , Therefore, using Chernoff bounds, probability of this event is bounded as . Under event , the buyer may end up with negative utility (as the strategy of always bidding reserve price will kick in), which can be at worst from this epoch. Otherwise, the buyer will get allocations, each from some rounds where . Since the buyer wins them at reserve price, and the space of other buyers’ bidding strategies consist only of functions that are independent of this buyer’s bid and valuation, the expected utility from each of these goods is .
Then, expected utility from each of of the next epochs is at least:
[TABLE]
∎
Upper bound on utility achievable by a bad state buyer.
Lemma 3**.**
Consider a buyer in bad state at the beginning of an epoch. Irrespective of the other buyers’ bids, its expected utility over the epoch is at most
[TABLE]
Proof.
For an epoch of length , the number of rounds of Bad-State-Auctions during an epoch is . During a bad state auction, the utility of a buyer with value is at most . Thus, the total expected utility of a buyer in bad state over the epoch is at most
[TABLE]
where the first and second inequality were obtained using that , and . The first part of third inequality used , and by definition , . The second part used . ∎
Appendix C Undominated Strategies for Buyers with Heterogeneous Lookahead Attitudes
We characterize the undominated strategies of myopic and lookahead buyers. The main lemma in this section, Lemma 4, argues that a -lookahead buyer, for large enough, playing an undominated strategy never enters a bad state. Lemma 8 then shows that in a bad state, myopic buyers playing undominated strategies always bid at least the reserve price when their value is at least .
Lookahead buyers.
We previously showed in Lemma 1 that we obtain the optimal revenue from buyers in a good state. In addition to obtaining the optimal revenue, we show that the mechanism also incentivizes lookahead buyers to stay in a good state. We denote the maximum length of an epoch by .
Lemma 4**.**
A -lookahead buyer with playing an undominated strategy never enters the bad state.
The mechanism moves a buyer to the bad state if a buyer bids lower than in a round of Good-State-Auctions where the number of previous uncleared auctions is . We show that during a round of Good-State-Auctions with , for every -lookahead buyer in good state , bidding below the reserve price is a dominated strategy.
A strategy which dominates bidding below the reserve price in such a case is the good strategy from Definition 12. Recall that if , this strategy bids if the valuation is greater than or equal to , and [math], otherwise; if the number of uncleared past auctions in this epoch is or more, it sets the bid equal to the reserve price for every value. Under this strategy, the buyer is guaranteed to always stay in good state or be moved to rest state, i.e., guaranteed to never be pushed to a bad state.
Lemma 5 and Lemma 6 upper and lower bound the -lookahead utility achieved by bidding below the reserve price and by playing during a round of Good-State-Auctions with . Then, by combining these bounds, we get that bidding below the reserve price in such a situation is dominated and obtain Lemma 4. We denote by the epoch at which step occurs.
Lemma 5**.**
Consider a round during Good-State-Auctions where the number of previous uncleared auctions is . Then, for any buyer and strategy which bids lower than in this round,
[TABLE]
Proof.
By the definition of Good-State-Auctions, bidder is moved to bad state at the end of step if it bids lower than . Since bidder bids lower than the reserve price, its utility in this round is [math].
Therefore, under this strategy, irrespective of the bids used in the rounds after , the buyer ’s -lookahead utility in round is at most the bad state utility over epochs, which by Lemma 3, is at most
[TABLE]
∎
Next, we lower bound the -lookahead utility achieved with the good strategy in the same situation.
Lemma 6**.**
Consider a round during Good-State-Auctions where the number of previous uncleared auctions is . Then, for any buyer playing strategy ,
[TABLE]
Proof.
The utility from bad state auctions is always non-negative, so we ignore that for the lower bound. Now, since any buyer can be allocated at most items in an epoch, the maximum payment in the remaining rounds of this epoch is at most , lower bounding the utility by . Now, consider the utility in any of the next epochs. The buyer always remains in good state or rest state in each of the next epochs. Combining the above argument with Lemma 12, the -lookahead utility of the new strategy is at least:
[TABLE]
∎
The third and last lemma needed for the proof of Lemma 4 is used to combine the two previous lemmas and argue that Good-State-Auctions lead higher utility for a buyer then Bad-State-Auctions. This lemma will also be used to argue that Good-State-Auctions give higher revenue.
Lemma 7**.**
For any such that and , then
[TABLE]
Proof.
First, for any , we have
[TABLE]
We obtain
[TABLE]
where the first inequality is since , the second is since is increasing in , and the last since . ∎
We now prove that entering a bad state is a dominated strategy for lookahead buyers by comparing the bounds obtained by Lemma 5 and Lemma 6.
Proof of Lemma 4.
With , we have . Comparing the -lookahead utility bounds obtained in Lemma 5 and Lemma 6, the strategy is dominating if
[TABLE]
We show the following three inequalities that when combined give the above inequality:
[TABLE]
We first show inequality (4). By Lemma 7, we have for all . Inequality 4 then holds by the assumption that .
For inequalities (5) and (6), we first have
[TABLE]
Inequality (5) then holds since . For inequality (6), note that
[TABLE]
where the second inequality is by Lemma 7 and that . ∎
Lemma 8**.**
Any myopic buyer playing an undominated strategy bids if its valuation is in any round of Bad-State-Auctions where is in bad state.
Proof.
Let be the set of bad state buyers in a round of Bad-State-Auctions of epoch . We argue that for every myopic buyer that has private valuation , bidding below is a dominated strategy. A myopic buyer maximizes its utility in the current round . With bid , buyer gets zero utility in round . If , bidding is dominated by the strategy of bidding , which always obtains non-negative utility in round and obtains strictly positive utility when other buyers bid below . ∎
Appendix D Strategies of No-regret Buyers with Heterogeneous Learning Behavior
Lemma 9**.**
Every no-policy regret learner must remain in good state in all but rounds.
Proof.
Here, we use as benchmark expert strategy of bidding in good state auctions whenever initially, and bidding continuously once uncleared auctions reach the limit ().
This bidding strategy ensures that the buyer is always in a good state, and achieves an expected utility of at least during epoch by Lemma 12. Since , the expected per round utility is at least
[TABLE]
during that epoch, for some . Therefore, since the class contains such sequences of single experts, the policy-regret learning buyer must achieve at least utility per round. Now, once in bad state, the buyer can achieve at most
[TABLE]
utility on average by Lemma 3 for some such that . By Lemma 7, for any such that and , then
Therefore, if , then the number of bad state epochs in buyer’s state trajectory can be at most . This implies that the learner remains in a good state for at least rounds. ∎
We use the set of experts as defined in Definition 10 in the lemmas below.
Lemma 10**.**
Consider as the set of rounds where buyer is participates in a bad state auction, and for all other buyers in bad state. If buyer is a no-regret learner against the set of experts in Definition 10, then the buyer must bid for all but of rounds in .
Proof.
Consider a bidding expert function defined as when the projected history indicates that the buyer is in the bad state and valuation ; and [math] otherwise. This is (or is arbitrarily close in case of discretization) one of the experts in the class of experts that the buyer is using. Consider any round where . On bidding reserve price in this round, if bids for all the other buyers, then this buyer wins the auction. For a given trajectory of valuations, and other buyers’ bids, denotes the set of such rounds in Bad-State-Auctions; specifically, . Therefore, the hindsight utility of this expert is at least
[TABLE]
Since the buyer is using a no-regret learning algorithm, she must be achieving a utility that is within of the above utility for every trajectory. Now, the buyer cannot make any positive utility in a bad state in round if . Therefore, on any given trajectory, a no-regret learning buyer must have bid in all but of rounds in . ∎
A corollary of the above lemma is that if all buyers are learning buyers at most of bad state auctions where some participating no-regret buyer has valuation above can go uncleared.
Corollary 1**.**
Consider the set of rounds among bad state auctions where all buyers in bad state are no-regret learners against experts , and at least one buyer in bad state has valuation of or more. Then, in all but such rounds, at least one buyer with is guaranteed to bid .
Appendix E Proof of the Main Result
We prove the main result, which is a bound on the revenue obtained by the mechanism under a heterogeneous buyer population consisting of an unknown proportion of -lookahead buyers for different , myopic buyers, no-policy regret learners, and no-regret learners
Theorem 1**.**
Let be the number buyers which are either -lookahead, for , or no-policy regret learners. Let be the number of remaining buyers which are either myopic or no-regret learners. Assume Algorithm 1 is instantiated with parameters s.t. . If the myopic and lookahead buyers play undominated strategies and the learners play no-regret or no-policy regret strategies, then the expected per round revenue is at least
[TABLE]
Here, , with being the quantile for the valuation distribution, and . is the optimal revenue in a single-item auction with buyers.
The following corollary can be obtained by simple algebraic manipulations of the result stated in the above theorem.
See 11
Proof of Theorem 11.
Let be the number of buyers which are either -lookahead with or no-policy regret learners and be the remaining buyers (myopic or no-regret learner). We define to be the collection of rounds where there is either a no-policy regret learner that is in bad state, a no-regret learner that bids when .
Consider an epoch where none of the rounds during that epoch are in and where no buyer is moved from good state to bad state. Let and denote the number of good state and bad state buyers in the beginning of this epoch.
By Lemma 4, all -lookahead buyers (for ) remain in good or rest state in all epochs. Since none of the rounds during epoch are in , every no-policy regret learner remains in good state in epoch by definition of . Therefore, and for some . By Lemma 1, we get that the expected total revenue from Good-State-Auctions in this epoch is at least .
For Bad-State-Auctions, we first recall that there is no-policy regret learner in bad state at epoch . By Lemma 8, every myopic buyer bids at least if during a round of Bad-State-Auctions. By definition of , ever no-regret learner bids at least if during a round of Bad-State-Auctions at epoch . by Lemma 2, the expected revenue of the mechanism from Bad-State-Auctions at epoch is thus at least .
The expected total revenue from this epoch is thus at least
[TABLE]
To bound this expected revenue, we consider two cases.
If .
In this case, . Since , we get
[TABLE]
Since and , this implies that
[TABLE]
Next, since and , we have
[TABLE]
Since , we conclude that the total expected revenue from this epoch in this case is at least
[TABLE]
where the inequality is since .
If .
In this case, we have . Wet get
[TABLE]
Since and , this implies that
[TABLE]
Thus, the expected total revenue from this epoch in this case is at least
[TABLE]
Since , we have . Since , in both cases, the expected per round revenue of epoch is thus at least
[TABLE]
where if
We considered an epoch where none of the rounds during that epoch are in and where no buyer was moved from good state to bad state. First, since once a buyer is moved to bad state they remain in bad state for every future epoch, there are at most epochs where a buyer is moved from good state to bad state. We discount revenue for each of those epochs. Second, by Lemma 9 and 10, . Thus, the number of epochs where there is at least one round during that epoch that is in is . We discount revenue for each of those epochs. We obtain the lower bound on per-round revenue as stated in the theorem statement. ∎
Appendix F Regarding Existence of Policy Regret Learning Algorithm
In general, achieving policy regret is difficult; [ADT12] show that there exists an adaptive adversary such that any learning algorithm has regret at least . However, it sufficient for us to consider policy regret learners against a small set of experts (e.g., size) and further when other buyers are restricted to not use history beyond past steps. Under such restrictions a simple learning algorithm is to initially explores every expert for steps and then use the expert with best performance for remaining time steps. We can show that this simple learning algorithm achieves policy regret under our mechanism with a small modification of resetting the mechanism once after steps so that this initial exploration does not hurt buyer’s utility in future rounds. This modification does not affect any of the revenue guarantees provided by our mechanism. Further discussion is provided in Appendix E.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[ACBFS 03] Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM J. Comput. , 32(1):48–77, January 2003.
- 2[ada] adage.com. Google’s ad manager will move to first-price auction. https://adage.com/article/digital/google-adx-moving-a-price-auction/316894/ .
- 3[ade] adexchanger.com. Rubicon joins first-price auction club; diageo is latest brand to demand more transparency. https://adexchanger.com/ad-exchange-news/tuesday-12122017// .
- 4[ADH 16] Itai Ashlagi, Constantinos Daskalakis, and Nima Haghpanah. Sequential mechanisms with ex-post participation guarantees. In Proceedings of the 2016 ACM Conference on Economics and Computation, EC ’16, Maastricht, The Netherlands, July 24-28, 2016 , pages 213–214, 2016.
- 5[ADMS 18] Shipra Agrawal, Constantinos Daskalakis, Vahab S. Mirrokni, and Balasubramanian Sivan. Robust repeated auctions under heterogeneous buyer behavior. In Proceedings of the 2018 ACM Conference on Economics and Computation, Ithaca, NY, USA, June 18-22, 2018 , page 171, 2018.
- 6[ADT 12] Raman Arora, Ofer Dekel, and Ambuj Tewari. Online bandit learning against an adaptive adversary: from regret to policy regret. In ICML . icml.cc / Omnipress, 2012.
- 7[AS 13] Susan Athey and Ilya Segal. An efficient dynamic mechanism. Econometrica , 81(6):2463–2485, 2013.
- 8[Bat 05] Marco Battaglini. Long-term contracting with markovian consumers. American Economic Review , 95(3):637–658, 2005.
