EST: Towards Efficient Scaling Laws in Click-Through Rate Prediction via Unified Modeling

TL;DR

This paper introduces EST, a unified transformer-based model for CTR prediction that processes raw inputs without lossy aggregation, enabling scalable performance improvements and practical deployment benefits.

Contribution

The paper proposes EST, a fully unified modeling approach with novel modules, achieving efficient scaling and improved industrial CTR prediction performance.

Findings

EST exhibits a stable power-law scaling relationship.

Outperforms production baselines with 3.27% RPM increase.

Achieves 1.22% CTR lift in industrial deployment.

Abstract

Efficiently scaling industrial Click-Through Rate (CTR) prediction has recently attracted significant research attention. Existing approaches typically employ early aggregation of user behaviors to maintain efficiency. However, such non-unified or partially unified modeling creates an information bottleneck by discarding fine-grained, token-level signals essential for unlocking scaling gains. In this work, we revisit the fundamental distinctions between CTR prediction and Large Language Models (LLMs), identifying two critical properties: the asymmetry in information density between behavioral and non-behavioral features, and the modality-specific priors of content-rich signals. Accordingly, we propose the Efficiently Scalable Transformer (EST), which achieves fully unified modeling by processing all raw inputs in a single sequence without lossy aggregation. EST integrates two modules: Lightweight Cross-Attention (LCA), which prunes redundant self-interactions to focus on high-impact cross-feature dependencies, and Content Sparse Attention (CSA), which utilizes content similarity to dynamically select high-signal behaviors. Extensive experiments show that EST exhibits a stable and efficient power-law scaling relationship, enabling predictable performance gains with model scale. Deployed on Taobao's display advertising platform, EST significantly outperforms production baselines, delivering a 3.27\% RPM (Revenue Per Mile) increase and a 1.22\% CTR lift, establishing a practical pathway for scalable industrial CTR prediction models.