HyMoERec: Hybrid Mixture-of-Experts for Sequential Recommendation

TL;DR

HyMoERec introduces a hybrid mixture-of-experts model for sequential recommendation, effectively capturing user and item heterogeneity, leading to improved performance over existing methods.

Contribution

The paper presents a novel hybrid mixture-of-experts architecture with adaptive fusion for sequential recommendation, addressing limitations of uniform models.

Findings

Outperforms state-of-the-art baselines on MovieLens-1M and Beauty datasets.

Effectively models diverse user behaviors and item complexities.

Ensures stable training with a hybrid expert design.

Abstract

We propose HyMoERec, a novel sequential recommendation framework that addresses the limitations of uniform Position-wise Feed-Forward Networks in existing models. Current approaches treat all user interactions and items equally, overlooking the heterogeneity in user behavior patterns and diversity in item complexity. HyMoERec initially introduces a hybrid mixture-of-experts architecture that combines shared and specialized expert branches with an adaptive expert fusion mechanism for the sequential recommendation task. This design captures diverse reasoning for varied users and items while ensuring stable training. Experiments on MovieLens-1M and Beauty datasets demonstrate that HyMoERec consistently outperforms state-of-the-art baselines.