Dreaming User Multimodal Representation Guided by The Platonic Representation Hypothesis for Micro-Video Recommendation

TL;DR

This paper introduces DreamUMM, a multimodal user representation method inspired by the Platonic Representation Hypothesis, which improves micro-video recommendations by capturing dynamic user interests in a shared multimodal space, validated through large-scale online tests.

Contribution

We propose DreamUMM, a novel real-time multimodal user representation approach based on a closed-form solution, and Candidate-DreamUMM for cold-start scenarios, demonstrating practical scalability and effectiveness.

Findings

Significant improvements in user engagement metrics.

Successful deployment on platforms with hundreds of millions of users.

Empirical evidence supporting multimodal interest convergence.

Abstract

The proliferation of online micro-video platforms has underscored the necessity for advanced recommender systems to mitigate information overload and deliver tailored content. Despite advancements, accurately and promptly capturing dynamic user interests remains a formidable challenge. Inspired by the Platonic Representation Hypothesis, which posits that different data modalities converge towards a shared statistical model of reality, we introduce DreamUMM (Dreaming User Multi-Modal Representation), a novel approach leveraging user historical behaviors to create real-time user representation in a multimoda space. DreamUMM employs a closed-form solution correlating user video preferences with multimodal similarity, hypothesizing that user interests can be effectively represented in a unified multimodal space. Additionally, we propose Candidate-DreamUMM for scenarios lacking recent user behavior data, inferring interests from candidate videos alone. Extensive online A/B tests demonstrate significant improvements in user engagement metrics, including active days and play count. The successful deployment of DreamUMM in two micro-video platforms with hundreds of millions of daily active users, illustrates its practical efficacy and scalability in personalized micro-video content delivery. Our work contributes to the ongoing exploration of representational convergence by providing empirical evidence supporting the potential for user interest representations to reside in a multimodal space.