LLM4Rerank: LLM-based Auto-Reranking Framework for Recommendations

TL;DR

This paper presents LLM4Rerank, a novel large language model-based reranking framework that effectively balances multiple criteria like accuracy, diversity, and fairness in recommender systems, while ensuring scalability and personalization.

Contribution

The paper introduces a comprehensive, scalable reranking framework using LLMs with a graph structure and Chain-of-Thought process for multi-criteria integration and personalization.

Findings

Outperforms existing reranking models on public datasets

Effectively balances accuracy, diversity, and fairness

Maintains scalability and personalization in recommendations

Abstract

Reranking is a critical component in recommender systems, playing an essential role in refining the output of recommendation algorithms. Traditional reranking models have focused predominantly on accuracy, but modern applications demand consideration of additional criteria such as diversity and fairness. Existing reranking approaches often fail to harmonize these diverse criteria effectively at the model level. Moreover, these models frequently encounter challenges with scalability and personalization due to their complexity and the varying significance of different reranking criteria in diverse scenarios. In response, we introduce a comprehensive reranking framework enhanced by LLM, designed to seamlessly integrate various reranking criteria while maintaining scalability and facilitating personalized recommendations. This framework employs a fully connected graph structure, allowing the LLM to simultaneously consider multiple aspects such as accuracy, diversity, and fairness through a coherent Chain-of-Thought (CoT) process. A customizable input mechanism is also integrated, enabling the tuning of the language model's focus to meet specific reranking needs. We validate our approach using three popular public datasets, where our framework demonstrates superior performance over existing state-of-the-art reranking models in balancing multiple criteria.