The Whole is Better than the Sum: Using Aggregated Demonstrations in In-Context Learning for Sequential Recommendation

TL;DR

This paper introduces LLMSRec-Syn, a novel aggregation method for demonstrations in in-context learning with LLMs, significantly improving sequential recommendation accuracy over existing methods.

Contribution

Proposes LLMSRec-Syn, an aggregation-based demonstration technique that enhances LLMs' performance in sequential recommendation tasks.

Findings

LLMSRec-Syn outperforms state-of-the-art LLM-based methods.

Aggregation of multiple demonstrations improves recommendation accuracy.

In some cases, matches or exceeds supervised learning methods.

Abstract

Large language models (LLMs) have shown excellent performance on various NLP tasks. To use LLMs as strong sequential recommenders, we explore the in-context learning approach to sequential recommendation. We investigate the effects of instruction format, task consistency, demonstration selection, and number of demonstrations. As increasing the number of demonstrations in ICL does not improve accuracy despite using a long prompt, we propose a novel method called LLMSRec-Syn that incorporates multiple demonstration users into one aggregated demonstration. Our experiments on three recommendation datasets show that LLMSRec-Syn outperforms state-of-the-art LLM-based sequential recommendation methods. In some cases, LLMSRec-Syn can perform on par with or even better than supervised learning methods. Our code is publicly available at https://github.com/demoleiwang/LLMSRec_Syn.