GAS: Generative Auto-bidding with Post-training Search

TL;DR

This paper introduces GAS, a flexible auto-bidding framework that refines generative models using post-training search techniques, improving bid strategies and adaptability across various advertiser preferences in online advertising.

Contribution

The paper proposes a novel GAS framework that combines weak-to-strong search alignment with a transformer-based voting mechanism, enabling effective preference adaptation without retraining multiple models.

Findings

Achieved 4.60% increase in target cost in real-world experiments.

Demonstrated effectiveness of search-based refinement in online A/B testing.

Enhanced model generalization across diverse advertiser preferences.

Abstract

Auto-bidding is essential in facilitating online advertising by automatically placing bids on behalf of advertisers. Generative auto-bidding, which generates bids based on an adjustable condition using models like transformers and diffusers, has recently emerged as a new trend due to its potential to learn optimal strategies directly from data and adjust flexibly to preferences. However, generative models suffer from low-quality data leading to a mismatch between the condition, like return to go, and true action value, especially in long sequential decision-making. Besides, the majority preference in the dataset may hinder models' generalization ability on minority advertisers' preferences. While it is possible to collect high-quality data and retrain multiple models for different preferences, the high cost makes it unaffordable, hindering the advancement of auto-bidding into the era of large foundation models. To address this, we propose a flexible and practical Generative Auto-bidding scheme using post-training Search, termed GAS, to refine a base policy model's output and adapt to various preferences. We use weak-to-strong search alignment by training small critics for different preferences and an MCTS-inspired search to refine the model's output. Specifically, a novel voting mechanism with transformer-based critics trained with policy indications could enhance search alignment performance. Additionally, utilizing the search, we provide a fine-tuning method for high-frequency preference scenarios considering computational efficiency. Extensive experiments conducted on the real-world dataset and online A/B test on the Kuaishou advertising platform demonstrate the effectiveness of GAS, achieving significant improvements, e.g., 4.60% increment of target cost.