Pyramid Mixer: Multi-dimensional Multi-period Interest Modeling for Sequential Recommendation

TL;DR

Pyramid Mixer is a novel sequential recommendation model that uses a pyramid-structured MLP-Mixer architecture to comprehensively model user interests across behaviors, features, and time periods, improving recommendation effectiveness.

Contribution

It introduces a pyramid-structured MLP-Mixer based model for multi-dimensional, multi-period user interest modeling in sequential recommendation systems.

Findings

Achieved +0.106% improvement in user stay duration.

Increased user active days by +0.0113%.

Successfully deployed in an industrial platform.

Abstract

Sequential recommendation, a critical task in recommendation systems, predicts the next user action based on the understanding of the user's historical behaviors. Conventional studies mainly focus on cross-behavior modeling with self-attention based methods while neglecting comprehensive user interest modeling for more dimensions. In this study, we propose a novel sequential recommendation model, Pyramid Mixer, which leverages the MLP-Mixer architecture to achieve efficient and complete modeling of user interests. Our method learns comprehensive user interests via cross-behavior and cross-feature user sequence modeling. The mixer layers are stacked in a pyramid way for cross-period user temporal interest learning. Through extensive offline and online experiments, we demonstrate the effectiveness and efficiency of our method, and we obtain a +0.106% improvement in user stay duration and a +0.0113% increase in user active days in the online A/B test. The Pyramid Mixer has been successfully deployed on the industrial platform, demonstrating its scalability and impact in real-world applications.