GLoSS: Generative Language Models with Semantic Search for Sequential Recommendation

TL;DR

GLoSS introduces a semantic search-based generative recommendation framework using quantized LLaMA models, achieving state-of-the-art results on Amazon datasets and outperforming existing LLM-based recommenders.

Contribution

The paper presents GLoSS, a novel generative recommendation approach combining semantic search with efficient LLMs, improving performance and robustness over prior methods.

Findings

Achieves state-of-the-art performance on Amazon datasets.

Outperforms traditional ID-based and LLM-based recommenders.

Excels in cold-start and long-user-history scenarios.

Abstract

We propose Generative Low-rank language model with Semantic Search (GLoSS), a generative recommendation framework that combines large language models with dense retrieval for sequential recommendation. Unlike prior methods such as GPT4Rec, which rely on lexical matching via BM25, GLoSS uses semantic search to retrieve relevant items beyond lexical matching. For query generation, we employ 4-bit quantized LlaMA-3 models fine-tuned with low-rank adaptation (LoRA), enabling efficient training and inference on modest hardware. We evaluate GLoSS on three real-world Amazon review datasets: Beauty, Toys, and Sports, and find that it achieves state-of-the-art performance. Compared to traditional ID-based baselines, GLoSS improves Recall@5 by 33.3%, 52.8%, and 15.2%, and NDCG@5 by 30.0%, 42.6%, and 16.1%, respectively. It also outperforms LLM-based recommenders such as P5, GPT4Rec, LlamaRec and E4SRec with Recall@5 gains of 4.3%, 22.8%, and 29.5%. Additionally, user segment evaluations show that GLoSS performs particularly well for cold-start users in the Amazon Toys and Sports datasets, and benefits from longer user histories in Amazon Beauty dataset, demonstrating robustness across different levels of interaction lengths.