Beyond Inter-Item Relations: Dynamic Adaption for Enhancing LLM-Based Sequential Recommendation

TL;DR

This paper introduces DARec, an LLM-based sequential recommendation model that enhances understanding of item relations, incorporates collaborative knowledge, and dynamically adapts architecture for improved recommendation accuracy.

Contribution

The paper presents a novel DARec model that integrates intra-item relation modeling, collaborative knowledge injection, and dynamic layer-wise adaptation for LLM-based sequential recommendation.

Findings

DARec outperforms existing models in recommendation accuracy.

The dynamic adaptation mechanism improves model flexibility and performance.

Intra-item relation modeling enhances understanding of token and item semantics.

Abstract

Sequential recommender systems (SRS) predict the next items that users may prefer based on user historical interaction sequences. Inspired by the rise of large language models (LLMs) in various AI applications, there is a surge of work on LLM-based SRS. Despite their attractive performance, existing LLM-based SRS still exhibit some limitations, including neglecting intra-item relations, ignoring long-term collaborative knowledge and using inflexible architecture designs for adaption. To alleviate these issues, we propose an LLM-based sequential recommendation model named DARec. Built on top of coarse-grained adaption for capturing inter-item relations, DARec is further enhanced with (1) context masking that models intra-item relations to help LLM better understand token and item semantics in the context of SRS, (2) collaborative knowledge injection that helps LLM incorporate long-term collaborative knowledge, and (3) a dynamic adaption mechanism that uses Bayesian optimization to flexibly choose layer-wise adapter architectures in order to better incorporate different sequential information. Extensive experiments demonstrate that DARec can effectively handle sequential recommendation in a dynamic and adaptive manner.