Beyond Semantic Understanding: Preserving Collaborative Frequency Components in LLM-based Recommendation

TL;DR

This paper introduces FreLLM4Rec, a spectral method that preserves collaborative signals in LLM-based recommenders, improving recommendation accuracy by balancing semantic and collaborative information.

Contribution

It proposes a novel spectral approach with G-LPF and TFM to maintain collaborative signals in LLM recommenders, addressing their tendency to weaken such signals.

Findings

Achieves up to 8% NDCG@10 improvement over baselines.

Effectively mitigates collaborative signal attenuation in LLM recommenders.

Provides theoretical guarantees for collaborative signal preservation.

Abstract

Recommender systems in concert with Large Language Models (LLMs) present promising avenues for generating semantically-informed recommendations. However, LLM-based recommenders exhibit a tendency to overemphasize semantic correlations within users' interaction history. When taking pretrained collaborative ID embeddings as input, LLM-based recommenders progressively weaken the inherent collaborative signals as the embeddings propagate through LLM backbones layer by layer, as opposed to traditional Transformer-based sequential models in which collaborative signals are typically preserved or even enhanced for state-of-the-art performance. To address this limitation, we introduce FreLLM4Rec, an approach designed to balance semantic and collaborative information from a spectral perspective. Item embeddings that incorporate both semantic and collaborative information are first purified using a Global Graph Low-Pass Filter (G-LPF) to preliminarily remove irrelevant high-frequency noise. Temporal Frequency Modulation (TFM) then actively preserves collaborative signal layer by layer. Note that the collaborative preservation capability of TFM is theoretically guaranteed by establishing a connection between the optimal but hard-to-implement local graph fourier filters and the suboptimal yet computationally efficient frequency-domain filters. Extensive experiments on four benchmark datasets demonstrate that FreLLM4Rec successfully mitigates collaborative signal attenuation and achieves competitive performance, with improvements of up to 8.00\% in NDCG@10 over the best baseline. Our findings provide insights into how LLMs process collaborative information and offer a principled approach for improving LLM-based recommendation systems.