Impression Allocation and Policy Search in Display Advertising

TL;DR

This paper introduces a novel auction-based framework and a multi-agent reinforcement learning approach to optimize impression allocation between guaranteed contracts and RTB in display advertising, improving revenue and stability.

Contribution

It formulates impression allocation as an auction problem and develops a scalable RL method to adapt bids under traffic fluctuations, achieving optimal outcomes.

Findings

Optimal bidding functions derived for guaranteed contracts.

The RL method effectively adapts to traffic changes.

Experimental results show improved revenue and stability.

Abstract

In online display advertising, guaranteed contracts and real-time bidding (RTB) are two major ways to sell impressions for a publisher. For large publishers, simultaneously selling impressions through both guaranteed contracts and in-house RTB has become a popular choice. Generally speaking, a publisher needs to derive an impression allocation strategy between guaranteed contracts and RTB to maximize its overall outcome (e.g., revenue and/or impression quality). However, deriving the optimal strategy is not a trivial task, e.g., the strategy should encourage incentive compatibility in RTB and tackle common challenges in real-world applications such as unstable traffic patterns (e.g., impression volume and bid landscape changing). In this paper, we formulate impression allocation as an auction problem where each guaranteed contract submits virtual bids for individual impressions. With this formulation, we derive the optimal bidding functions for the guaranteed contracts, which result in the optimal impression allocation. In order to address the unstable traffic pattern challenge and achieve the optimal overall outcome, we propose a multi-agent reinforcement learning method to adjust the bids from each guaranteed contract, which is simple, converging efficiently and scalable. The experiments conducted on real-world datasets demonstrate the effectiveness of our method.