GRAB: An LLM-Inspired Sequence-First Click-Through Rate Prediction Modeling Paradigm

TL;DR

GRAB is a novel generative CTR prediction framework inspired by LLMs that effectively models user behavior sequences, outperforming traditional DLRMs in online deployment with significant revenue and CTR improvements.

Contribution

It introduces a new generative modeling paradigm for CTR prediction, incorporating a causal multi-channel attention mechanism inspired by LLMs, and demonstrates scalable performance improvements.

Findings

Outperforms existing DLRMs in online deployment with 3.05% revenue increase.

Achieves a 3.49% rise in CTR.

Exhibits linear scaling with longer user interaction sequences.

Abstract

Traditional Deep Learning Recommendation Models (DLRMs) face increasing bottlenecks in performance and efficiency, often struggling with generalization and long-sequence modeling. Inspired by the scaling success of Large Language Models (LLMs), we propose Generative Ranking for Ads at Baidu (GRAB), an end-to-end generative framework for Click-Through Rate (CTR) prediction. GRAB integrates a novel Causal Action-aware Multi-channel Attention (CamA) mechanism to effectively capture temporal dynamics and specific action signals within user behavior sequences. Full-scale online deployment demonstrates that GRAB significantly outperforms established DLRMs, delivering a 3.05% increase in revenue and a 3.49% rise in CTR. Furthermore, the model demonstrates desirable scaling behavior: its expressive power shows a monotonic and approximately linear improvement as longer interaction sequences are utilized.