Tensor-based Sequential Learning via Hankel Matrix Representation for Next Item Recommendations

TL;DR

This paper introduces a tensor factorization model using Hankel matrix representation for next item recommendation, aiming to replicate self-attention dynamics with a simpler, linear architecture.

Contribution

It proposes a novel tensor-based approach that captures sequential data structure, providing a lightweight alternative to self-attention models for recommendation tasks.

Findings

Competitive performance with neural models

Reproduces properties of self-attention networks

Simplifies architecture to shallow linear model

Abstract

Self-attentive transformer models have recently been shown to solve the next item recommendation task very efficiently. The learned attention weights capture sequential dynamics in user behavior and generalize well. Motivated by the special structure of learned parameter space, we question if it is possible to mimic it with an alternative and more lightweight approach. We develop a new tensor factorization-based model that ingrains the structural knowledge about sequential data within the learning process. We demonstrate how certain properties of a self-attention network can be reproduced with our approach based on special Hankel matrix representation. The resulting model has a shallow linear architecture and compares competitively to its neural counterpart.