CaliCausalRank: Calibrated Multi-Objective Ad Ranking with Robust Counterfactual Utility Optimization

TL;DR

CaliCausalRank is a unified framework for multi-objective ad ranking that improves calibration, robustness, and utility estimation by integrating calibration, constrained optimization, and counterfactual methods, leading to better offline and online performance.

Contribution

It introduces a novel training-time calibration approach, constraint-based multi-objective optimization, and variance-reduced counterfactual estimators for ad ranking.

Findings

Achieves 1.1% relative AUC improvement.

Reduces calibration error by 31.6%.

Gains 3.2% in utility over baseline.

Abstract

Ad ranking systems must simultaneously optimize multiple objectives including click-through rate (CTR), conversion rate (CVR), revenue, and user experience metrics. However, production systems face critical challenges: score scale inconsistency across traffic segments undermines threshold transferability, and position bias in click logs causes offline-online metric discrepancies. We propose CaliCausalRank, a unified framework that integrates training-time scale calibration, constraint-based multi-objective optimization, and robust counterfactual utility estimation. Our approach treats score calibration as a first-class training objective rather than post-hoc processing, employs Lagrangian relaxation for constraint satisfaction, and utilizes variance-reduced counterfactual estimators for reliable offline evaluation. Experiments on the Criteo and Avazu datasets demonstrate that CaliCausalRank achieves 1.1% relative AUC improvement, 31.6% calibration error reduction, and 3.2% utility gain compared to the best baseline (PairRank) while maintaining consistent performance across different traffic segments.