Causality-Enhanced Behavior Sequence Modeling in LLMs for Personalized Recommendation

TL;DR

This paper introduces a counterfactual fine-tuning method for LLMs that enhances personalized recommendation by explicitly modeling the causal influence of user behavior sequences, leading to improved recommendation accuracy.

Contribution

It proposes a novel counterfactual fine-tuning approach with token-level weighting to better leverage behavior sequences in LLM-based recommender systems.

Findings

CFT improves behavior sequence modeling in LLMs.

Enhanced recommendation accuracy demonstrated on real-world datasets.

Token-level weighting effectively captures diminishing influence of earlier tokens.

Abstract

Recent advancements in recommender systems have focused on leveraging Large Language Models (LLMs) to improve user preference modeling, yielding promising outcomes. However, current LLM-based approaches struggle to fully leverage user behavior sequences, resulting in suboptimal preference modeling for personalized recommendations. In this study, we propose a novel Counterfactual Fine-Tuning (CFT) method to address this issue by explicitly emphasizing the role of behavior sequences when generating recommendations. Specifically, we employ counterfactual reasoning to identify the causal effects of behavior sequences on model output and introduce a task that directly fits the ground-truth labels based on these effects, achieving the goal of explicit emphasis. Additionally, we develop a token-level weighting mechanism to adjust the emphasis strength for different item tokens, reflecting the diminishing influence of behavior sequences from earlier to later tokens during predicting an item. Extensive experiments on real-world datasets demonstrate that CFT effectively improves behavior sequence modeling. Our codes are available at https://github.com/itsmeyjt/CFT.