Crocodile: Cross Experts Covariance for Disentangled Learning in Multi-Domain Recommendation

TL;DR

Crocodile introduces a covariance loss for disentangled multi-domain learning, effectively capturing diverse user interests and addressing data imbalance, leading to improved recommendation performance and online advertising metrics.

Contribution

The paper proposes a novel covariance loss for disentangled learning in multi-domain recommendation, enabling better domain-specific feature learning despite data imbalance.

Findings

Outperforms state-of-the-art methods on public datasets

Achieves 0.72% CTR lift in online A/B testing

Achieves 0.73% GMV lift in online A/B testing

Abstract

Multi-domain learning (MDL) has become a prominent topic in enhancing the quality of personalized services. It's critical to learn commonalities between domains and preserve the distinct characteristics of each domain. However, this leads to a challenging dilemma in MDL. On the one hand, a model needs to leverage domain-aware modules such as experts or embeddings to preserve each domain's distinctiveness. On the other hand, real-world datasets often exhibit long-tailed distributions across domains, where some domains may lack sufficient samples to effectively train their specific modules. Unfortunately, nearly all existing work falls short of resolving this dilemma. To this end, we propose a novel Cross-experts Covariance Loss for Disentangled Learning model (Crocodile), which employs multiple embedding tables to make the model domain-aware at the embeddings which consist most parameters in the model, and a covariance loss upon these embeddings to disentangle them, enabling the model to capture diverse user interests among domains. Empirical analysis demonstrates that our method successfully addresses both challenges and outperforms all state-of-the-art methods on public datasets. During online A/B testing in Tencent's advertising platform, Crocodile achieves 0.72% CTR lift and 0.73% GMV lift on a primary advertising scenario.