Learning Positional Attention for Sequential Recommendation

TL;DR

This paper investigates learned positional embeddings in self-attention models for sequential recommendation, proposing new models that explicitly learn positional relations, resulting in improved performance over existing methods.

Contribution

It introduces novel attention models, PARec and FPARec, that directly learn positional relations, enhancing sequential recommendation accuracy.

Findings

PARec and FPARec outperform previous self-attention models

Learned positional embeddings often encode token distances

Proposed models achieve state-of-the-art results in experiments

Abstract

Self-attention-based networks have achieved remarkable performance in sequential recommendation tasks. A crucial component of these models is positional encoding. In this study, we delve into the learned positional embedding, demonstrating that it often captures the distance between tokens. Building on this insight, we introduce novel attention models that directly learn positional relations. Extensive experiments reveal that our proposed models, \textbf{PARec} and \textbf{FPARec} outperform previous self-attention-based approaches. The code can be found here: https://github.com/NetEase-Media/FPARec.