Hyena Operator for Fast Sequential Recommendation

TL;DR

HyenaRec introduces a scalable, efficient sequential recommender that combines polynomial kernel parameterization with gated convolutions, outperforming traditional attention-based models especially on long user sequences.

Contribution

The paper presents HyenaRec, a novel hybrid architecture using polynomial kernels and gating to improve long-sequence recommendation efficiency and accuracy.

Findings

HyenaRec outperforms baselines in ranking accuracy across datasets.

It trains up to 6 times faster than attention-based models.

Maintains efficiency and accuracy on long user sequences.

Abstract

Sequential recommendation models, particularly those based on attention, achieve strong accuracy but incur quadratic complexity, making long user histories prohibitively expensive. Sub-quadratic operators such as Hyena provide efficient alternatives in language modeling, but their potential in recommendation remains underexplored. We argue that Hyena faces challenges in recommendation due to limited representation capacity on sparse, long user sequences. To address these challenges, we propose HyenaRec, a novel sequential recommender that integrates polynomial-based kernel parameterization with gated convolutions. Specifically, we design convolutional kernels using Legendre orthogonal polynomials, which provides a smooth and compact basis for modeling long-term temporal dependencies. A complementary gating mechanism captures fine-grained short-term behavioral bursts, yielding a hybrid architecture that balances global temporal evolution with localized user interests under sparse feedback. This construction enhances expressiveness while scaling linearly with sequence length. Extensive experiments on multiple real-world datasets demonstrate that HyenaRec consistently outperforms Attention-, Recurrent-, and other baselines in ranking accuracy. Moreover, it trains significantly faster (up to 6x speedup), with particularly pronounced advantages on long-sequence scenarios where efficiency is maintained without sacrificing accuracy. These results highlight polynomial-based kernel parameterization as a principled and scalable alternative to attention for sequential recommendation.