Designing a Sequential Recommendation System for Heterogeneous Interactions Using Transformers

TL;DR

This paper introduces a Transformer-based sequential recommendation system that accounts for heterogeneous interaction types and their dependency relationships, aiming to improve recommendation accuracy in scenarios like educational content sequencing.

Contribution

It proposes a novel Transformer architecture that models multiple event types and their dependencies, advancing sequential recommendation methods beyond traditional RNN-based approaches.

Findings

Enhanced recommendation accuracy with heterogeneous event modeling

Effective incorporation of event dependency relationships

Outperforms existing sequential recommendation models

Abstract

While many production-ready and robust algorithms are available for the task of recommendation systems, many of these systems do not take the order of user's consumption into account. The order of consumption can be very useful and matters in many scenarios. One such scenario is an educational content recommendation, where users generally follow a progressive path towards more advanced courses. Researchers have used RNNs to build sequential recommendation systems and other models that deal with sequences. Sequential Recommendation systems try to predict the next event for the user by reading their history. With the massive success of Transformers in Natural Language Processing and their usage of Attention Mechanism to better deal with sequences, there have been attempts to use this family of models as a base for a new generation of sequential recommendation systems. In this work, by converting each user's interactions with items into a series of events and basing our architecture on Transformers, we try to enable the use of such a model that takes different types of events into account. Furthermore, by recognizing that some events have to occur before some other types of events take place, we try to modify the architecture to reflect this dependency relationship and enhance the model's performance.