Generate, Not Recommend: Personalized Multimodal Content Generation

TL;DR

This paper introduces a novel approach that uses large multimodal models to generate personalized content, such as images, directly tailored to individual users, surpassing traditional recommendation systems in creating novel, relevant items.

Contribution

It proposes a new paradigm of content generation for personalization using multimodal models trained with supervised and reinforcement learning, enabling the creation of tailored content beyond filtering existing items.

Findings

Generated images align with user preferences.

Models produce relevant content for future interests.

Method outperforms traditional recommendation approaches.

Abstract

To address the challenge of information overload from massive web contents, recommender systems are widely applied to retrieve and present personalized results for users. However, recommendation tasks are inherently constrained to filtering existing items and lack the ability to generate novel concepts, limiting their capacity to fully satisfy user demands and preferences. In this paper, we propose a new paradigm that goes beyond content filtering and selecting: directly generating personalized items in a multimodal form, such as images, tailored to individual users. To accomplish this, we leverage any-to-any Large Multimodal Models (LMMs) and train them in both supervised fine-tuning and online reinforcement learning strategy to equip them with the ability to yield tailored next items for users. Experiments on two benchmark datasets and user study confirm the efficacy of the proposed method. Notably, the generated images not only align well with users' historical preferences but also exhibit relevance to their potential future interests.