LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendation

TL;DR

This paper introduces LLMEmb, a novel approach using large language models to generate item embeddings that improve sequential recommendation systems, especially for long-tail, low-popularity items, through specialized fine-tuning and adaptation techniques.

Contribution

The paper proposes LLMEmb, a new method that leverages LLMs with supervised contrastive fine-tuning and collaborative signal integration to enhance recommendation performance.

Findings

LLMEmb significantly outperforms existing methods on real-world datasets.

The approach effectively addresses the long-tail problem in SRS.

Embeddings generated by LLMEmb improve recommendation accuracy across multiple models.

Abstract

Sequential Recommender Systems (SRS), which model a user's interaction history to predict the next item of interest, are widely used in various applications. However, existing SRS often struggle with low-popularity items, a challenge known as the long-tail problem. This issue leads to reduced serendipity for users and diminished profits for sellers, ultimately harming the overall system. Large Language Model (LLM) has the ability to capture semantic relationships between items, independent of their popularity, making it a promising solution to this problem. In this paper, we introduce LLMEmb, a novel method leveraging LLM to generate item embeddings that enhance SRS performance. To bridge the gap between general-purpose LLM and the recommendation domain, we propose a Supervised Contrastive Fine-Tuning (SCFT) approach. This approach includes attribute-level data augmentation and a tailored contrastive loss to make LLM more recommendation-friendly. Additionally, we emphasize the importance of integrating collaborative signals into LLM-generated embeddings, for which we propose Recommendation Adaptation Training (RAT). This further refines the embeddings for optimal use in SRS. The LLMEmb-derived embeddings can be seamlessly integrated with any SRS models, underscoring the practical value. Comprehensive experiments conducted on three real-world datasets demonstrate that LLMEmb significantly outperforms existing methods across multiple SRS models. The code for our method is released online https://github.com/Applied-Machine-Learning-Lab/LLMEmb.