Heterogeneous User Modeling for LLM-based Recommendation

TL;DR

This paper introduces HUM, a novel method leveraging LLMs with compression and robustness enhancers to improve open-domain recommendation by effectively modeling heterogeneous user behaviors across multiple domains.

Contribution

HUM is the first approach to incorporate a compression and robustness framework specifically designed for heterogeneous user modeling in LLM-based recommendation systems.

Findings

HUM outperforms existing methods in accuracy and robustness.

The compression enhancer effectively condenses heterogeneous behaviors.

The domain importance score mitigates the domain seesaw phenomenon.

Abstract

Leveraging Large Language Models (LLMs) for recommendation has demonstrated notable success in various domains, showcasing their potential for open-domain recommendation. A key challenge to advancing open-domain recommendation lies in effectively modeling user preferences from users' heterogeneous behaviors across multiple domains. Existing approaches, including ID-based and semantic-based modeling, struggle with poor generalization, an inability to compress noisy interactions effectively, and the domain seesaw phenomenon. To address these challenges, we propose a Heterogeneous User Modeling (HUM) method, which incorporates a compression enhancer and a robustness enhancer for LLM-based recommendation. The compression enhancer uses a customized prompt to compress heterogeneous behaviors into a tailored token, while a masking mechanism enhances cross-domain knowledge extraction and understanding. The robustness enhancer introduces a domain importance score to mitigate the domain seesaw phenomenon by guiding domain optimization. Extensive experiments on heterogeneous datasets validate that HUM effectively models user heterogeneity by achieving both high efficacy and robustness, leading to superior performance in open-domain recommendation.