FLAME: Condensing Ensemble Diversity into a Single Network for Efficient Sequential Recommendation

TL;DR

FLAME is a novel framework that condenses ensemble diversity into a single network for efficient sequential recommendation, achieving high performance with reduced computational cost.

Contribution

FLAME introduces a modular ensemble approach with a frozen and a learnable network, enabling ensemble-level diversity in a single network for recommendation tasks.

Findings

Outperforms state-of-the-art baselines on six datasets.

Achieves up to 7.69× faster convergence.

Improves NDCG@20 by 9.70%.

Abstract

Sequential recommendation requires capturing diverse user behaviors, which a single network often fails to capture. While ensemble methods mitigate this by leveraging multiple networks, training them all from scratch leads to high computational cost and instability from noisy mutual supervision. We propose {\bf F}rozen and {\bf L}earnable networks with {\bf A}ligned {\bf M}odular {\bf E}nsemble ({\bf FLAME}), a novel framework that condenses ensemble-level diversity into a single network for efficient sequential recommendation. During training, FLAME simulates exponential diversity using only two networks via {\it modular ensemble}. By decomposing each network into sub-modules (e.g., layers or blocks) and dynamically combining them, FLAME generates a rich space of diverse representation patterns. To stabilize this process, we pretrain and freeze one network to serve as a semantic anchor and employ {\it guided mutual learning}. This aligns the diverse representations into the space of the remaining learnable network, ensuring robust optimization. Consequently, at inference, FLAME utilizes only the learnable network, achieving ensemble-level performance with zero overhead compared to a single network. Experiments on six datasets show that FLAME outperforms state-of-the-art baselines, achieving up to 7.69$\times$ faster convergence and 9.70\% improvement in NDCG@20. We provide the source code of FLAME at https://github.com/woo-joo/FLAME_SIGIR26.