Mitigating Propensity Bias of Large Language Models for Recommender Systems

TL;DR

This paper introduces CLLMR, a framework that mitigates propensity bias in LLM-based recommender systems by embedding structural information and using counterfactual inference to improve recommendation quality.

Contribution

The paper proposes a novel spectrum-based encoder and counterfactual inference method to address bias and dimensional collapse in LLM-based recommenders, enhancing their effectiveness.

Findings

Consistently improves recommender performance across models

Effectively mitigates propensity bias and dimensional collapse

Enhances capturing of user preferences

Abstract

The rapid development of Large Language Models (LLMs) creates new opportunities for recommender systems, especially by exploiting the side information (e.g., descriptions and analyses of items) generated by these models. However, aligning this side information with collaborative information from historical interactions poses significant challenges. The inherent biases within LLMs can skew recommendations, resulting in distorted and potentially unfair user experiences. On the other hand, propensity bias causes side information to be aligned in such a way that it often tends to represent all inputs in a low-dimensional subspace, leading to a phenomenon known as dimensional collapse, which severely restricts the recommender system's ability to capture user preferences and behaviours. To address these issues, we introduce a novel framework named Counterfactual LLM Recommendation (CLLMR). Specifically, we propose a spectrum-based side information encoder that implicitly embeds structural information from historical interactions into the side information representation, thereby circumventing the risk of dimension collapse. Furthermore, our CLLMR approach explores the causal relationships inherent in LLM-based recommender systems. By leveraging counterfactual inference, we counteract the biases introduced by LLMs. Extensive experiments demonstrate that our CLLMR approach consistently enhances the performance of various recommender models.