Improving LLM-based Recommendation with Self-Hard Negatives from Intermediate Layers

TL;DR

This paper introduces ILRec, a novel fine-tuning framework for LLM-based recommender systems that leverages self-hard negative signals from intermediate layers to improve preference learning and recommendation accuracy.

Contribution

ILRec is the first to extract and utilize self-hard negative tokens from intermediate layers for more effective preference learning in LLM-based recommendation.

Findings

ILRec significantly outperforms existing methods on three datasets.

Self-hard negatives improve the discriminative power of preference models.

The framework effectively balances negative signal quality with false negative mitigation.

Abstract

Large language models (LLMs) have shown great promise in recommender systems, where supervised fine-tuning (SFT) is commonly used for adaptation. Subsequent studies further introduce preference learning to incorporate negative samples into the training process. However, existing methods rely on sequence-level, offline-generated negatives, making them less discriminative and informative when adapting LLMs to recommendation tasks with large negative item spaces. To address these challenges, we propose ILRec, a novel preference fine-tuning framework for LLM-based recommendation, leveraging self-hard negative signals extracted from intermediate layers to improve preference learning. Specifically, we identify self-hard negative tokens from intermediate layers as fine-grained negative supervision that dynamically reflects the model's preference learning process. To effectively integrate these signals into training, we design a two-stage framework comprising cross-layer preference optimization and cross-layer preference distillation, enabling the model to jointly discriminate informative negatives and enhance the quality of negative signals from intermediate layers. In addition, we introduce a lightweight collaborative filtering model to assign token-level rewards for negative signals, mitigating the risk of over-penalizing false negatives. Extensive experiments on three datasets demonstrate ILRec's effectiveness in enhancing the performance of LLM-based recommender systems.