All Roads Lead to Rome: Unveiling the Trajectory of Recommender Systems Across the LLM Era

TL;DR

This paper reviews the evolution of recommender systems, emphasizing the integration of large language models (LLMs) and their potential to enhance personalization through list-wise and conversational recommendation paths.

Contribution

It provides a comprehensive overview of recommender system evolution, focusing on LLM integration, and identifies key research directions and challenges for future personalization technologies.

Findings

Two main evolution paths: list-wise and conversational recommendation.

LLMs enable improved long-term memory and reasoning in recommender systems.

Challenges include developing effective personalization interfaces and addressing inherent technical issues.

Abstract

Recommender systems (RS) are vital for managing information overload and delivering personalized content, responding to users' diverse information needs. The emergence of large language models (LLMs) offers a new horizon for redefining recommender systems with vast general knowledge and reasoning capabilities. Standing across this LLM era, we aim to integrate recommender systems into a broader picture, and pave the way for more comprehensive solutions for future research. Therefore, we first offer a comprehensive overview of the technical progression of recommender systems, particularly focusing on language foundation models and their applications in recommendation. We identify two evolution paths of modern recommender systems -- via list-wise recommendation and conversational recommendation. These two paths finally converge at LLM agents with superior capabilities of long-term memory, reflection, and tool intelligence. Along these two paths, we point out that the information effectiveness of the recommendation is increased, while the user's acquisition cost is decreased. Technical features, research methodologies, and inherent challenges for each milestone along the path are carefully investigated -- from traditional list-wise recommendation to LLM-enhanced recommendation to recommendation with LLM agents. Finally, we highlight several unresolved challenges crucial for the development of future personalization technologies and interfaces and discuss the future prospects.