Break the ID-Language Barrier: An Adaption Framework for LLM-based Sequential Recommendation

TL;DR

This paper introduces IDLE-Adapter, a framework that enhances large language models for sequential recommendation by integrating domain-specific ID embeddings, leading to significant improvements in recommendation accuracy.

Contribution

The paper presents a novel framework that effectively incorporates ID embeddings into LLMs for sequential recommendation, addressing domain knowledge gaps and demonstrating high flexibility.

Findings

Over 10% improvement in HitRate@5

Over 20% improvement in NDCG@5

Effective integration across diverse models and architectures

Abstract

The recent breakthrough of large language models (LLMs) in natural language processing has sparked exploration in recommendation systems, however, their limited domain-specific knowledge remains a critical bottleneck. Specifically, LLMs lack key pieces of information crucial for sequential recommendations, such as user behavior patterns. To address this critical gap, we propose IDLE-Adapter, a novel framework that integrates pre-trained ID embeddings, rich in domain-specific knowledge, into LLMs to improve recommendation accuracy. IDLE-Adapter acts as a bridge, transforming sparse user-item interaction data into dense, LLM-compatible representations through a Pre-trained ID Sequential Model, Dimensionality Alignment, Layer-wise Embedding Refinement, and Layer-wise Distribution Alignment. Furthermore, IDLE-Adapter demonstrates remarkable flexibility by seamlessly integrating ID embeddings from diverse ID-based sequential models and LLM architectures. Extensive experiments across various datasets demonstrate the superiority of IDLE-Adapter, achieving over 10\% and 20\% improvements in HitRate@5 and NDCG@5 metrics, respectively, compared to state-of-the-art methods.