Solving the Content Gap in Roblox Game Recommendations: LLM-Based Profile Generation and Reranking

TL;DR

This paper leverages large language models to generate structured game features from in-game text and re-rank recommendations, significantly improving personalization and relevance in Roblox's user-generated content ecosystem.

Contribution

It introduces a scalable LLM-based framework for extracting game attributes and re-ranking recommendations, addressing challenges of sparse and inconsistent game text data.

Findings

Enhanced recommendation relevance through LLM-generated features

Improved personalization and user satisfaction in Roblox recommendations

Deployed in production for user engagement and integrity detection

Abstract

With the vast and dynamic user-generated content on Roblox, creating effective game recommendations requires a deep understanding of game content. Traditional recommendation models struggle with the inconsistent and sparse nature of game text features such as titles and descriptions. Recent advancements in large language models (LLMs) offer opportunities to enhance recommendation systems by analyzing in-game text data. This paper addresses two challenges: generating high-quality, structured text features for games without extensive human annotation, and validating these features to ensure they improve recommendation relevance. We propose an approach that extracts in-game text and uses LLMs to infer attributes such as genre and gameplay objectives from raw player interactions. Additionally, we introduce an LLM-based re-ranking mechanism to assess the effectiveness of the generated text features, enhancing personalization and user satisfaction. Beyond recommendations, our approach supports applications such as user engagement-based integrity detection, already deployed in production. This scalable framework demonstrates the potential of in-game text understanding to improve recommendation quality on Roblox and adapt recommendations to its unique, user-generated ecosystem.