Functional Optimization Reinforcement Learning for Real-Time Bidding

TL;DR

This paper introduces a multi-agent reinforcement learning framework with functional optimization for real-time bidding, significantly enhancing campaign profitability and efficiency in simulated auction environments.

Contribution

It proposes a novel multi-agent RL architecture with functional optimization for RTB, including four agents with different attributes, improving bidding performance.

Findings

Agents with functional optimization achieved higher winning rates.

The approach improved campaign profitability in simulated tests.

Functional actions and rewards enhanced bidding effectiveness.

Abstract

Real-time bidding is the new paradigm of programmatic advertising. An advertiser wants to make the intelligent choice of utilizing a \textbf{Demand-Side Platform} to improve the performance of their ad campaigns. Existing approaches are struggling to provide a satisfactory solution for bidding optimization due to stochastic bidding behavior. In this paper, we proposed a multi-agent reinforcement learning architecture for RTB with functional optimization. We designed four agents bidding environment: three Lagrange-multiplier based functional optimization agents and one baseline agent (without any attribute of functional optimization) First, numerous attributes have been assigned to each agent, including biased or unbiased win probability, Lagrange multiplier, and click-through rate. In order to evaluate the proposed RTB strategy's performance, we demonstrate the results on ten sequential simulated auction campaigns. The results show that agents with functional actions and rewards had the most significant average winning rate and winning surplus, given biased and unbiased winning information respectively. The experimental evaluations show that our approach significantly improve the campaign's efficacy and profitability.