Double-Wing Mixture of Experts for Streaming Recommendations

TL;DR

This paper introduces VRS-DWMoE, a novel framework combining variational sampling and a double-wing mixture of experts to enhance streaming recommendation accuracy by capturing user preference drift and heterogeneity.

Contribution

It proposes a new sampling method and a double-wing MoE model to better learn and adapt to long-term and heterogeneous user preferences in streaming recommendations.

Findings

VRS-DWMoE outperforms state-of-the-art SRSs in accuracy.

The framework effectively captures long-term user preferences.

It handles data heterogeneity better than existing models.

Abstract

Streaming Recommender Systems (SRSs) commonly train recommendation models on newly received data only to address user preference drift, i.e., the changing user preferences towards items. However, this practice overlooks the long-term user preferences embedded in historical data. More importantly, the common heterogeneity in data stream greatly reduces the accuracy of streaming recommendations. The reason is that different preferences (or characteristics) of different types of users (or items) cannot be well learned by a unified model. To address these two issues, we propose a Variational and Reservoir-enhanced Sampling based Double-Wing Mixture of Experts framework, called VRS-DWMoE, to improve the accuracy of streaming recommendations. In VRS-DWMoE, we first devise variational and reservoir-enhanced sampling to wisely complement new data with historical data, and thus address the user preference drift issue while capturing long-term user preferences. After that, we propose a Double-Wing Mixture of Experts (DWMoE) model to first effectively learn heterogeneous user preferences and item characteristics, and then make recommendations based on them. Specifically, DWMoE contains two Mixture of Experts (MoE, an effective ensemble learning model) to learn user preferences and item characteristics, respectively. Moreover, the multiple experts in each MoE learn the preferences (or characteristics) of different types of users (or items) where each expert specializes in one underlying type. Extensive experiments demonstrate that VRS-DWMoE consistently outperforms the state-of-the-art SRSs.