Expressiveness Limits of Autoregressive Semantic ID Generation in Generative Recommendation

TL;DR

This paper investigates the limitations of autoregressive generative recommendation models caused by their structured decoding space and proposes a modification called Latte to improve their expressiveness.

Contribution

The paper reveals how tree-structured decoding constrains model expressiveness and introduces Latte, a simple method that reshapes the decoding space to enhance recommendation performance.

Findings

Structural correlations in autoregressive models hinder distinguishing similar items.

Latte improves NDCG@10 by an average of 3.45%.

Reshaping decoding trees relaxes probability coupling among items.

Abstract

Generative recommendation (GR) models generate items by autoregressively producing a sequence of discrete tokens that jointly index the target item. However, this autoregressive generation process also induces a structured decoding space whose impact on model expressiveness remains underexplored. Specifically, token-by-token generation can be viewed as traversing a decoding tree induced by semantic ID tokens, where leaf nodes correspond to candidate items. We observe that the item probabilities produced by GR models are strongly correlated with this tree structure: items that are close in the tree tend to receive similar probabilities for any given user, making it difficult to distinguish among them based on user-specific preferences. We further show theoretically that such structural correlations prevent GR models from representing even simple patterns that can be well captured by conventional collaborative filtering models. To mitigate this issue, we propose Latte, a simple modification that injects a latent token before each semantic ID, reshaping the decoding space from a single tree into multiple latent-token-conditioned trees. This design creates multiple paths with varying tree distances between items, relaxing tree-induced probability coupling and yielding an average of 3.45% relative improvement on NDCG@10. Our code is available at https://github.com/hyp1231/Latte.