Distillation-based Scenario-Adaptive Mixture-of-Experts for the Matching Stage of Multi-scenario Recommendation

TL;DR

This paper introduces DSMOE, a novel distillation-based approach with a scenario-adaptive module for multi-scenario recommendation, enhancing matching accuracy especially in data-sparse scenarios.

Contribution

It proposes DSMOE, combining a lightweight scenario-adaptive projection with cross-architecture knowledge distillation to improve multi-scenario matching performance.

Findings

Significantly improves retrieval quality in data-sparse scenarios

Outperforms existing methods on real-world datasets

Effectively prevents expert collapse in long-tail scenarios

Abstract

Multi-scenario recommendation is pivotal for optimizing user experience across diverse contexts. While Multi-gate Mixture-of-Experts (MMOE) thrives in ranking, its transfer to the matching stage is hindered by the blind optimization inherent to independent two-tower architectures and the parameter dominance of head scenarios. To address these structural and distributional bottlenecks, we propose Distillation-based Scenario-Adaptive Mixture-of-Experts (DSMOE). Specially, we devise a Scenario-Adaptive Projection (SAP) module to generate lightweight, context-specific parameters, effectively preventing expert collapse in long-tail scenarios. Concurrently, we introduce a cross-architecture knowledge distillation framework, where an interaction-aware teacher guides the two-tower student to capture complex matching patterns. Extensive experiments on real-world datasets demonstrate DSMOE's superiority, particularly in significantly improving retrieval quality for under-represented, data-sparse scenarios.