Budget Optimization for Sponsored Search: Censored Learning in MDPs

TL;DR

This paper addresses budget optimization in sponsored search auctions by modeling it as a censored learning problem in MDPs, proposing a new algorithm, and validating its effectiveness on real-world data.

Contribution

It introduces a novel MDP framework with censored observations for budget optimization and develops a Kaplan-Meier-based learning algorithm.

Findings

The proposed algorithm converges quickly to optimal performance.

It outperforms several baseline methods on real search auction data.

Demonstrates effectiveness in maximizing clicks within budget constraints.

Abstract

We consider the budget optimization problem faced by an advertiser participating in repeated sponsored search auctions, seeking to maximize the number of clicks attained under that budget. We cast the budget optimization problem as a Markov Decision Process (MDP) with censored observations, and propose a learning algorithm based on the wellknown Kaplan-Meier or product-limit estimator. We validate the performance of this algorithm by comparing it to several others on a large set of search auction data from Microsoft adCenter, demonstrating fast convergence to optimal performance.