M3oE: Multi-Domain Multi-Task Mixture-of Experts Recommendation Framework

TL;DR

M3oE is an innovative recommendation framework that adaptively integrates multi-domain and multi-task information using mixture-of-experts modules and AutoML, achieving superior performance on benchmark datasets.

Contribution

The paper introduces M3oE, the first adaptive multi-domain multi-task recommendation framework leveraging mixture-of-experts and AutoML for dynamic structure optimization.

Findings

Outperforms diverse baselines on benchmark datasets

Effectively models complex dependencies among multiple domains and tasks

Demonstrates the benefits of adaptive, disentangled preference learning

Abstract

Multi-domain recommendation and multi-task recommendation have demonstrated their effectiveness in leveraging common information from different domains and objectives for comprehensive user modeling. Nonetheless, the practical recommendation usually faces multiple domains and tasks simultaneously, which cannot be well-addressed by current methods. To this end, we introduce M3oE, an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework. M3oE integrates multi-domain information, maps knowledge across domains and tasks, and optimizes multiple objectives. We leverage three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences respectively to address the complex dependencies among multiple domains and tasks in a disentangled manner. Additionally, we design a two-level fusion mechanism for precise control over feature extraction and fusion across diverse domains and tasks. The framework's adaptability is further enhanced by applying AutoML technique, which allows dynamic structure optimization. To the best of the authors' knowledge, our M3oE is the first effort to solve multi-domain multi-task recommendation self-adaptively. Extensive experiments on two benchmark datasets against diverse baselines demonstrate M3oE's superior performance. The implementation code is available to ensure reproducibility.