Bi-Level Optimization for Generative Recommendation: Bridging Tokenization and Generation

TL;DR

This paper introduces BLOGER, a bi-level optimization framework that jointly trains tokenizers and recommenders for generative recommendation, improving alignment and performance.

Contribution

It proposes a unified bi-level optimization approach with meta-learning and gradient surgery to better align tokenization with recommendation objectives.

Findings

BLOGER outperforms state-of-the-art methods on multiple datasets.

The framework maintains efficiency with no significant extra computational cost.

Joint training improves recommendation accuracy by aligning tokenization with recommendation goals.

Abstract

Generative recommendation is emerging as a transformative paradigm by directly generating recommended items, rather than relying on matching. Building such a system typically involves two key components: (1) optimizing the tokenizer to derive suitable item identifiers, and (2) training the recommender based on those identifiers. Existing approaches often treat these components separately--either sequentially or in alternation--overlooking their interdependence. This separation can lead to misalignment: the tokenizer is trained without direct guidance from the recommendation objective, potentially yielding suboptimal identifiers that degrade recommendation performance. To address this, we propose BLOGER, a Bi-Level Optimization for GEnerative Recommendation framework, which explicitly models the interdependence between the tokenizer and the recommender in a unified optimization process. The lower level trains the recommender using tokenized sequences, while the upper level optimizes the tokenizer based on both the tokenization loss and recommendation loss. We adopt a meta-learning approach to solve this bi-level optimization efficiently, and introduce gradient surgery to mitigate gradient conflicts in the upper-level updates, thereby ensuring that item identifiers are both informative and recommendation-aligned. Extensive experiments on multiple real-world datasets demonstrate that BLOGER consistently outperforms state-of-the-art generative recommendation methods while maintaining practical efficiency with no significant additional computational overhead, effectively bridging the gap between item tokenization and autoregressive generation. We release our code at https://github.com/Ten-Mao/BLOGER.