SODA: Semantic-Oriented Distributional Alignment for Generative Recommendation

TL;DR

SODA introduces a distribution-level supervision framework for generative recommendation, enhancing semantic alignment and performance by leveraging probabilistic distributions over codebooks, and enabling end-to-end training.

Contribution

It proposes a novel distributional supervision paradigm and a contrastive framework, SODA, for improved semantic alignment in generative recommenders, addressing limitations of discrete supervision.

Findings

SODA consistently improves recommendation accuracy across datasets.

The framework effectively aligns semantic distributions via negative KL divergence.

End-to-end training enhances the integration of tokenizer and recommender.

Abstract

Generative recommendation has emerged as a scalable alternative to traditional retrieve-and-rank pipelines by operating in a compact token space. However, existing methods mainly rely on discrete code-level supervision, which leads to information loss and limits the joint optimization between the tokenizer and the generative recommender. In this work, we propose a distribution-level supervision paradigm that leverages probability distributions over multi-layer codebooks as soft and information-rich representations. Building on this idea, we introduce Semantic-Oriented Distributional Alignment (SODA), a plug-and-play contrastive supervision framework based on Bayesian Personalized Ranking, which aligns semantically rich distributions via negative KL divergence while enabling end-to-end differentiable training. Extensive experiments on multiple real-world datasets demonstrate that SODA consistently improves the performance of various generative recommender backbones, validating its effectiveness and generality. Codes will be available upon acceptance.