Bridging the Gap: Self-Optimized Fine-Tuning for LLM-based Recommender Systems

TL;DR

This paper introduces SOFT, a novel self-optimized fine-tuning approach that combines guidance and tuning strategies with curriculum learning to significantly improve LLM-based recommender system performance.

Contribution

The paper proposes a new Self-Optimized Fine-Tuning method that effectively bridges the gap between LLM knowledge and recommendation tasks using self-distillation and curriculum learning.

Findings

SOFT improves recommendation accuracy by 37.59% on average.

Self-distillation creates an easy-to-learn dataset for LLMs.

Curriculum learning enables gradual adaptation to complex recommendation data.

Abstract

Recent years have witnessed extensive exploration of Large Language Models (LLMs) on the field of Recommender Systems (RS). There are currently two commonly used strategies to enable LLMs to have recommendation capabilities: 1) The "Guidance-Only" strategy uses in-context learning to exploit and amplify the inherent semantic understanding and item recommendation capabilities of LLMs; 2) The "Tuning-Only" strategy uses supervised fine-tuning (SFT) to fine-tune LLMs with the aim of fitting them to real recommendation data. However, neither of these strategies can effectively bridge the gap between the knowledge space of LLMs and recommendation, and their performance do not meet our expectations. To better enable LLMs to learn recommendation knowledge, we combine the advantages of the above two strategies and proposed a novel "Guidance+Tuning" method called Self-Optimized Fine-Tuning (SOFT), which adopts the idea of curriculum learning. It first employs self-distillation to construct an auxiliary easy-to-learn but meaningful dataset from a fine-tuned LLM. Then it further utilizes a self-adaptive curriculum scheduler to enable LLMs to gradually learn from simpler data (self-distilled data) to more challenging data (real RS data). Extensive experiments demonstrate that SOFT significantly enhances the recommendation accuracy (37.59\% on average) of LLM-based methods. The code is available via https://anonymous.4open.science/r/Self-Optimized-Fine-Tuning-264E