DenseRec: Revisiting Dense Content Embeddings for Sequential Transformer-based Recommendation

TL;DR

DenseRec introduces a dual-path embedding approach that effectively integrates dense content embeddings into transformer-based recommenders, improving cold-start performance without complex infrastructure.

Contribution

It proposes a simple method that learns a linear projection from dense content embeddings to ID embeddings, enabling better generalization to unseen items in sequential recommendation.

Findings

DenseRec outperforms ID-only SASRec on three real-world datasets.

The method improves sequence representations for unseen items.

DenseRec works well with compact embedding models without extra tuning.

Abstract

Transformer-based sequential recommenders, such as SASRec or BERT4Rec, typically rely solely on learned item ID embeddings, making them vulnerable to the item cold-start problem, particularly in environments with dynamic item catalogs. While dense content embeddings from pre-trained models offer potential solutions, direct integration into transformer-based recommenders has consistently underperformed compared to ID-only approaches. We revisit this integration challenge and propose DenseRec, a simple yet effective method that introduces a dual-path embedding approach. DenseRec learns a linear projection from the dense embedding space into the ID embedding space during training, enabling seamless generalization to previously unseen items without requiring specialized embedding models or complex infrastructure. In experiments on three real-world datasets, we find DenseRec to consistently outperform an ID-only SASRec baseline, even without additional hyperparameter tuning and while using compact embedding models. Our analysis suggests improvements primarily arise from better sequence representations in the presence of unseen items, positioning DenseRec as a practical and robust solution for cold-start sequential recommendation.