CELA: Cost-Efficient Language Model Alignment for CTR Prediction

TL;DR

CELA introduces a cost-efficient, model-agnostic framework that combines textual features and collaborative filtering for improved CTR prediction, demonstrating superior offline and online performance.

Contribution

The paper proposes CELA, a novel framework that aligns language models with ID-based models, enhancing CTR prediction with textual features while maintaining efficiency.

Findings

CELA outperforms state-of-the-art methods in offline experiments.

Online A/B testing shows CELA's practical effectiveness in real-world systems.

CELA maintains training and inference efficiency comparable to ID-based models.

Abstract

Click-Through Rate (CTR) prediction holds a paramount position in recommender systems. The prevailing ID-based paradigm underperforms in cold-start scenarios due to the skewed distribution of feature frequency. Additionally, the utilization of a single modality fails to exploit the knowledge contained within textual features. Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs). They design hard prompts to structure raw features into text for each interaction and then apply PLMs for text processing. With external knowledge and reasoning capabilities, PLMs extract valuable information even in cases of sparse interactions. Nevertheless, compared to ID-based models, pure text modeling degrades the efficacy of collaborative filtering, as well as feature scalability and efficiency during both training and inference. To address these issues, we propose \textbf{C}ost-\textbf{E}fficient \textbf{L}anguage Model \textbf{A}lignment (\textbf{CELA}) for CTR prediction. CELA incorporates textual features and language models while preserving the collaborative filtering capabilities of ID-based models. This model-agnostic framework can be equipped with plug-and-play textual features, with item-level alignment enhancing the utilization of external information while maintaining training and inference efficiency. Through extensive offline experiments, CELA demonstrates superior performance compared to state-of-the-art methods. Furthermore, an online A/B test conducted on an industrial App recommender system showcases its practical effectiveness, solidifying the potential for real-world applications of CELA.