Improving the Performance of Sequential Recommendation Systems with an Extended Large Language Model

TL;DR

This paper demonstrates that replacing Llama2 with Llama3 in a recommendation system significantly enhances performance across multiple datasets, leveraging recent advances in large language models for better personalization.

Contribution

The study introduces a simple yet effective method of improving recommendation system performance by updating the underlying LLM to a newer version without structural modifications.

Findings

Performance improvements of 38.65%, 8.69%, and 8.19% on ML-100K, Beauty, and Games datasets.

Model replacement yields cost-effective enhancement without system redesign.

Validates the practicality of using latest LLMs in recommendation systems.

Abstract

Recently, competition in the field of artificial intelligence (AI) has intensified among major technological companies, resulting in the continuous release of new large-language models (LLMs) that exhibit improved language understanding and context-based reasoning capabilities. It is expected that these advances will enable more efficient personalized recommendations in LLM-based recommendation systems through improved quality of training data and architectural design. However, many studies have not considered these recent developments. In this study, it was proposed to improve LLM-based recommendation systems by replacing Llama2 with Llama3 in the LlamaRec framework. To ensure a fair comparison, random seed values were set and identical input data was provided during preprocessing and training. The experimental results show average performance improvements of 38.65\%, 8.69\%, and 8.19\% for the ML-100K, Beauty, and Games datasets, respectively, thus confirming the practicality of this method. Notably, the significant improvements achieved by model replacement indicate that the recommendation quality can be improved cost-effectively without the need to make structural changes to the system. Based on these results, it is our contention that the proposed approach is a viable solution for improving the performance of current recommendation systems.