360Brew: A Decoder-only Foundation Model for Personalized Ranking and Recommendation

TL;DR

This paper introduces 360Brew V1.0, a large decoder-only foundation model that simplifies ranking and recommendation systems by using natural language interfaces, reducing feature engineering and model maintenance efforts.

Contribution

The paper presents a novel large decoder-only model capable of handling multiple recommendation tasks without task-specific fine-tuning, leveraging natural language interfaces for flexibility and efficiency.

Findings

Achieves comparable or better performance than existing systems on LinkedIn tasks.

Reduces need for feature engineering and complex model dependencies.

Handles over 30 predictive tasks across the platform.

Abstract

Ranking and recommendation systems are the foundation for numerous online experiences, ranging from search results to personalized content delivery. These systems have evolved into complex, multilayered architectures that leverage vast datasets and often incorporate thousands of predictive models. The maintenance and enhancement of these models is a labor intensive process that requires extensive feature engineering. This approach not only exacerbates technical debt but also hampers innovation in extending these systems to emerging problem domains. In this report, we present our research to address these challenges by utilizing a large foundation model with a textual interface for ranking and recommendation tasks. We illustrate several key advantages of our approach: (1) a single model can manage multiple predictive tasks involved in ranking and recommendation, (2) decoder models with textual interface due to their comprehension of reasoning capabilities, can generalize to new recommendation surfaces and out-of-domain problems, and (3) by employing natural language interfaces for task definitions and verbalizing member behaviors and their social connections, we eliminate the need for feature engineering and the maintenance of complex directed acyclic graphs of model dependencies. We introduce our research pre-production model, 360Brew V1.0, a 150B parameter, decoder-only model that has been trained and fine-tuned on LinkedIn's data and tasks. This model is capable of solving over 30 predictive tasks across various segments of the LinkedIn platform, achieving performance levels comparable to or exceeding those of current production systems based on offline metrics, without task-specific fine-tuning. Notably, each of these tasks is conventionally addressed by dedicated models that have been developed and maintained over multiple years by teams of a similar or larger size than our own.