SpecTran: Spectral-Aware Transformer-based Adapter for LLM-Enhanced Sequential Recommendation

TL;DR

SpecTran introduces a spectral-aware transformer adapter that leverages the full spectral information in item embeddings, significantly improving sequential recommendation performance over existing methods.

Contribution

It proposes a novel spectral domain attention mechanism with spectral-position encoding to enhance item embedding integration in SR models.

Findings

Achieves an average of 9.17% improvement across datasets.

Outperforms strong baselines with spectral-aware attention.

Effectively utilizes full spectral information for recommendation.

Abstract

Traditional sequential recommendation (SR) models learn low-dimensional item ID embeddings from user-item interactions, often overlooking textual information such as item titles or descriptions. Recent advances in Large Language Models (LLMs) have inspired a surge of research that encodes item textual information with high-dimensional semantic embeddings, and designs transformation methods to inject such embeddings into SR models. These embedding transformation strategies can be categorized into two types, both of which exhibits notable drawbacks: 1) adapter-based methods suffer from pronounced dimension collapse, concentrating information into a few dominant dimensions; 2) SVD-based methods are rigid and manual, considering only a few principal spectral components while discarding rich information in the remaining spectrum. To address these limitations, we propose SpecTran, a spectral-aware transformer-based adapter that operates in the spectral domain, attending to the full spectrum to select and aggregates informative components. A learnable spectral-position encoding injects singular-value cues as an inductive bias, guiding attention toward salient spectral components and promoting diversity across embedding dimensions. Across four real-world datasets and three SR backbones, it consistently outperforms strong baselines, achieving an average improvement of 9.17%.