Closing the Performance Gap in Generative Recommenders with Collaborative Tokenization and Efficient Modeling

TL;DR

This paper introduces COSETTE and MARIUS, novel methods that enhance generative recommender systems by integrating collaborative signals and improving efficiency, significantly narrowing the performance gap with traditional ID-based models.

Contribution

The paper presents COSETTE for collaborative tokenization and MARIUS for efficient generative modeling, advancing generative recommenders to match ID-based model performance.

Findings

COSETTE improves item representation quality.

MARIUS reduces inference cost and boosts accuracy.

Generative models close the performance gap with ID-based models.

Abstract

Recent work has explored generative recommender systems as an alternative to traditional ID-based models, reframing item recommendation as a sequence generation task over discrete item tokens. While promising, such methods often underperform in practice compared to well-tuned ID-based baselines like SASRec. In this paper, we identify two key limitations holding back generative approaches: the lack of collaborative signal in item tokenization, and inefficiencies in the commonly used encoder-decoder architecture. To address these issues, we introduce COSETTE, a contrastive tokenization method that integrates collaborative information directly into the learned item representations, jointly optimizing for both content reconstruction and recommendation relevance. Additionally, we propose MARIUS, a lightweight, audio-inspired generative model that decouples timeline modeling from item decoding. MARIUS reduces inference cost while improving recommendation accuracy. Experiments on standard sequential recommendation benchmarks show that our approach narrows, or even eliminates, the performance gap between generative and modern ID-based models, while retaining the benefits of the generative paradigm.