When Transformers Meet Recommenders: Integrating Self-Attentive Sequential Recommendation with Fine-Tuned LLMs

TL;DR

This paper introduces SASRecLLM, a hybrid recommendation framework that combines self-attentive sequential recommendation with fine-tuned large language models, enhancing recommendation quality especially in cold-start scenarios.

Contribution

It presents a novel modular architecture integrating SASRec with LLMs via a mapping layer and tailored training strategies, advancing LLM-based recommendation methods.

Findings

SASRecLLM outperforms strong baselines in multiple datasets.

The framework improves cold-start and warm-start recommendation accuracy.

Extensive experiments validate the robustness and effectiveness of the approach.

Abstract

Self-Attentive Sequential Recommendation (SASRec) effectively captures long-term user preferences by applying attention mechanisms to historical interactions. Concurrently, the rise of Large Language Models (LLMs) has motivated research into LLM-based recommendation, which leverages their powerful generalization and language understanding capabilities. However, LLMs often lack the domain-specific knowledge and collaborative signals essential for high-quality recommendations when relying solely on textual prompts. To address this limitation, this study proposes SASRecLLM, a novel framework that integrates SASRec as a collaborative encoder with an LLM fine-tuned using Low-Rank Adaptation (LoRA). The components are connected via a mapping layer to align their dimensional spaces, and three targeted training strategies are designed to optimize the hybrid architecture. Extensive experiments on multiple datasets demonstrate that SASRecLLM achieves robust and consistent improvements over strong baselines in both cold-start and warm-start scenarios. This work advances the field of LLM-based recommendation by presenting a modular and effective paradigm for fusing structured collaborative filtering with the semantic power of fine-tuned LLMs. The implementation is available on GitHub: https://github.com/kechenkristin/RecLLM