On Negative-aware Preference Optimization for Recommendation

TL;DR

This paper introduces NAPO, a novel negative-aware preference optimization framework for LLM-based recommendation systems that enhances accuracy and reduces bias by efficiently utilizing negative samples.

Contribution

NAPO's key innovations include in-batch negative sharing and dynamic reward margin adjustment, improving negative sample utilization without extra memory costs.

Findings

NAPO outperforms existing methods in recommendation accuracy.

NAPO effectively reduces popularity bias.

Experiments on three datasets validate NAPO's effectiveness.

Abstract

Recommendation systems leverage user interaction data to suggest relevant items while filtering out irrelevant (negative) ones. The rise of large language models (LLMs) has garnered increasing attention for their potential in recommendation tasks. However, existing methods for optimizing LLM-based recommenders face challenges in effectively utilizing negative samples. Simply integrating large numbers of negative samples can improve ranking accuracy and mitigate popularity bias but often leads to increased computational overhead and memory costs. Additionally, current approaches fail to account for the varying informativeness of negative samples, leading to suboptimal optimization performance. To address these issues, we propose NAPO (\textbf{N}egative-\textbf{A}ware \textbf{P}reference \textbf{O}ptimization), an enhanced framework for preference optimization in LLM-based recommendation. NAPO introduces two key innovations: (1) in-batch negative sharing, which expands the pool of negative samples without additional memory overhead, and (2) dynamic reward margin adjustment, which adapts model updates based on the confidence of negative samples. Extensive experiments on three public datasets demonstrate that NAPO outperforms existing methods in both recommendation accuracy and popularity bias reduction.