Act-With-Think: Chunk Auto-Regressive Modeling for Generative Recommendation

TL;DR

This paper introduces Chunk AutoRegressive Modeling (CAR), a novel generative recommendation framework that integrates semantics and user behavior into a unified autoregressive transformer, significantly improving recommendation accuracy.

Contribution

CAR is the first to incorporate semantics and behavior into a single autoregressive model for recommendation, emulating a 'think-and-act' process and enhancing prediction performance.

Findings

CAR outperforms traditional AR methods with up to 22.30% improvement in Recall@5.

Scaling the number of semantics bits improves model performance.

CAR demonstrates a slow-thinking mechanism similar to reasoning in large language models.

Abstract

Generative recommendation (GR) typically encodes behavioral or semantic aspects of item information into discrete tokens, leveraging the standard autoregressive (AR) generation paradigm to make predictions. However, existing methods tend to overlook their intrinsic relationship, that is, the semantic usually provides some reasonable explainability "$\textbf{why}$" for the behavior "$\textbf{what}$", which may constrain the full potential of GR. To this end, we present Chunk AutoRegressive Modeling (CAR), a new generation paradigm following the decision pattern that users usually think semantic aspects of items (e.g. brand) and then take actions on target items (e.g. purchase). Our CAR, for the $\textit{first time}$, incorporates semantics (SIDs) and behavior (UID) into a single autoregressive transformer from an ``act-with-think'' dual perspective via chunk-level autoregression. Specifically, CAR packs SIDs and UID into a conceptual chunk for item unified representation, allowing each decoding step to make a holistic prediction. Experiments show that our CAR significantly outperforms existing methods based on traditional AR, improving Recall@5 by 7.93% to 22.30%. Furthermore, we verify the scaling effect between model performance and SIDs bit number, demonstrating that CAR preliminary emulates a kind of slow-thinking style mechanism akin to the reasoning processes observed in large language models (LLMs).