Reasoning While Recommending: Entropy-Guided Latent Reasoning in Generative Re-ranking Models

TL;DR

This paper introduces EGLR, a novel entropy-guided latent reasoning model for generative re-ranking, enabling real-time reasoning during list generation to better adapt to dynamic entropy changes and improve recommendation accuracy.

Contribution

It proposes a lightweight, entropy-guided reasoning mechanism that integrates reasoning during recommendation, enhancing exploration-exploitation balance without complex modifications.

Findings

EGLR reduces entropy in decision-making processes.

Experimental validation shows improved recommendation performance.

Model is compatible with existing generative re-ranking systems.

Abstract

Reinforcement learning plays a crucial role in generative re-ranking scenarios due to its exploration-exploitation capabilities, but existing generative methods mostly fail to adapt to the dynamic entropy changes in model difficulty during list generation, making it challenging to accurately capture complex preferences. Given that language models have achieved remarkable breakthroughs by integrating reasoning capabilities, we draw on this approach to introduce a latent reasoning mechanism, and experimental validation demonstrates that this mechanism effectively reduces entropy in the model's decision-making process. Based on these findings, we introduce the Entropy-Guided Latent Reasoning (EGLR) recommendation model, which has three core advantages. First, it abandons the "reason first, recommend later" paradigm to achieve "reasoning while recommending", specifically designed for the high-difficulty nature of list generation by enabling real-time reasoning during generation. Second, it implements entropy-guided variable-length reasoning using context-aware reasoning token alongside dynamic temperature adjustment, expanding exploration breadth in reasoning and boosting exploitation precision in recommending to achieve a more precisely adapted exploration-exploitation trade-off. Third, the model adopts a lightweight integration design with no complex independent modules or post-processing, enabling easy adaptation to existing models. Experimental results on two real-world datasets validate the model's effectiveness, and its notable advantage lies in being compatible with existing generative re-ranking models to enhance their performance. Further analyses also demonstrate its practical deployment value and research potential.