Pacer and Runner: Cooperative Learning Framework between Single- and Cross-Domain Sequential Recommendation

TL;DR

This paper introduces a cooperative learning framework for cross-domain sequential recommendation that adaptively manages negative transfer and enhances recommendation accuracy, demonstrated by significant improvements in real-world applications.

Contribution

The proposed model estimates negative transfer per domain and adaptively weights domain contributions, integrating mutual information maximization for improved cross-domain recommendation performance.

Findings

Outperforms previous models on two real-world datasets.

Achieves a 21.4% increase in click-through rate in a deployed system.

Effectively manages negative transfer across multiple domains.

Abstract

Cross-Domain Sequential Recommendation (CDSR) improves recommendation performance by utilizing information from multiple domains, which contrasts with Single-Domain Sequential Recommendation (SDSR) that relies on a historical interaction within a specific domain. However, CDSR may underperform compared to the SDSR approach in certain domains due to negative transfer, which occurs when there is a lack of relation between domains or different levels of data sparsity. To address the issue of negative transfer, our proposed CDSR model estimates the degree of negative transfer of each domain and adaptively assigns it as a weight factor to the prediction loss, to control gradient flows through domains with significant negative transfer. To this end, our model compares the performance of a model trained on multiple domains (CDSR) with a model trained solely on the specific domain (SDSR) to evaluate the negative transfer of each domain using our asymmetric cooperative network. In addition, to facilitate the transfer of valuable cues between the SDSR and CDSR tasks, we developed an auxiliary loss that maximizes the mutual information between the representation pairs from both tasks on a per-domain basis. This cooperative learning between SDSR and CDSR tasks is similar to the collaborative dynamics between pacers and runners in a marathon. Our model outperformed numerous previous works in extensive experiments on two real-world industrial datasets across ten service domains. We also have deployed our model in the recommendation system of our personal assistant app service, resulting in 21.4% increase in click-through rate compared to existing models, which is valuable to real-world business.