MLoRA: Multi-Domain Low-Rank Adaptive Network for CTR Prediction

TL;DR

MLoRA introduces a multi-domain low-rank adaptation method for CTR prediction, significantly improving performance across diverse domains and demonstrating effectiveness in real-world Alibaba deployment.

Contribution

The paper proposes a novel multi-domain low-rank adaptive network (MLoRA) with specialized LoRA modules for each domain, enhancing multi-domain CTR prediction efficiency and accuracy.

Findings

Achieves significant performance improvements over state-of-the-art baselines.

Demonstrates effectiveness and flexibility in Alibaba's real-world environment.

Code is publicly available for reproducibility.

Abstract

Click-through rate (CTR) prediction is one of the fundamental tasks in the industry, especially in e-commerce, social media, and streaming media. It directly impacts website revenues, user satisfaction, and user retention. However, real-world production platforms often encompass various domains to cater for diverse customer needs. Traditional CTR prediction models struggle in multi-domain recommendation scenarios, facing challenges of data sparsity and disparate data distributions across domains. Existing multi-domain recommendation approaches introduce specific-domain modules for each domain, which partially address these issues but often significantly increase model parameters and lead to insufficient training. In this paper, we propose a Multi-domain Low-Rank Adaptive network (MLoRA) for CTR prediction, where we introduce a specialized LoRA module for each domain. This approach enhances the model's performance in multi-domain CTR prediction tasks and is able to be applied to various deep-learning models. We evaluate the proposed method on several multi-domain datasets. Experimental results demonstrate our MLoRA approach achieves a significant improvement compared with state-of-the-art baselines. Furthermore, we deploy it in the production environment of the Alibaba.COM. The online A/B testing results indicate the superiority and flexibility in real-world production environments. The code of our MLoRA is publicly available.